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This  article  presents  the  findings  of  a  study  that  investigated 
the  relationships  between  self-leadership  and  creativity  in  the 
context  of  a  defense  acquisition  organization  and  its  employees. 
More  specifically,  this  study  examined  differences  in  self¬ 
leadership,  creativity,  and  perceived  organizational  support 
for  creativity  between  line-  and  supervisory-level  defense 
acquisition  employees.  The  resultant  analyses  suggested  that 
self-leadership  was  significantly  related  to  creative  potential 
and  practiced  creativity  for  both  line-  and  supervisory-level 
employees,  although  there  were  no  significant  differences  in 
overall  levels  of  self-leadership  between  the  two  groups.  In 
addition,  the  study  findings  revealed  significant  differences 
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perceptions  of  organizational  support  for  creativity. 
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uncertainty  of  a  project’s  future  and  funding  cutbacks  are 
found  to  have  a  strong  predictive  influence  on  development 
program  effectiveness,  which  may  be  explained  in  part 
by  their  impact  on  program  staffing  turnover  and  the 


disruption  of  testing  regimes.  A  central  conclusion  from 
this  study  is  that  shorter  development  cycle  times  favorably 
correlate  with  lower  levels  of  these  sources  of  program 
instability,  and  with  substantially  better  project  outcomes. 
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disconnect  between  the  use  of  certain  measures  like 
incentive  and  award  fees  and  expected  outcomes  in  major 
acquisitions  (GAO,  2005).  In  response  to  the  GAO  report, 
the  authors  conducted  a  research  effort  to  better  understand 
where  incentive  and  award  fees  had  favorable  impact  on 
performance  outcomes  and  why.  This  article  summarizes  the 
findings  of  the  research,  highlights  several  organizations  that 
clearly  used  techniques  that  drove  favorable  outcomes,  and 
provides  recommendations  and  take-aways  that  will  promote 
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The  author  applies  social  construction  theory  to  reveal  potential 
blind  spots  associated  with  the  technical  rationality  paradigm 
rooted  in  the  Defense  Department’s  Planning,  Programming, 
Budgeting  and  Execution  (PPBE)  process.  Drawing  heavily 
on  Karl  E.  Weick's  (1995)  version  of  sensemaking  (i.e.,  using, 
modifying,  rejecting,  and  creating  new  paradigms  or  shared 
mental  models  when  dealing  with  situations  of  incoherency 
and  disorderliness),  this  article  reveals  the  paradoxical 
qualities  along  three  socially  constructed  schemes:  the 
Pentagon  world  of  PPBE,  the  world  of  political  reasoning, 
and  the  hyperturbulent  world  described  as  our  contemporary 
operating  environment  (COE).  Ultimately,  the  author  argues 
that  Defense  resource  management  professionals  could 
be  living  in  a  dream  world  not  unlike  that  imagined  by 
the  character,  Neo,  in  the  popular  movie,  The  Matrix©. 
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REFORM  FOR  EMERGING  NEEDS 

COR  Stott  Moran,  USN 

When  the  Defense  Acquisition  Performance  Assessment 
(DAPA)  panel  proposed  sweeping  reforms  to  address 
long-standing  problems  in  defense  acquisition,  their 
recommendations  did  not  anticipate  critical  challenges 
expected  in  the  development  of  a  System  of  Systems  (SoS). 
Defense  leaders  counting  on  revolutionary  SoS  capabilities 
must  appreciate  that  current  and  proposed  acquisition 
systems  insufficiently  facilitate  SoS  development.  This 
article  describes  the  importance  of  adapting  defense 
acquisition  processes  to  enable  effective  SoS  development 
and  concludes  with  proposed  modifications  to  the  DAPA 
Report  recommendations.  Tailoring  defense  acquisition 
organization,  budgeting,  and  requirements  generation  systems 
to  overcome  the  challenges  of  SoS  acquisition  will  be  essential 
for  tomorrow’s  military  systems  to  realize  their  potential. 
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A  GLIMPSE  INTO  DoD  WEAPON  SYSTEMS  PROGRAMS 

Andy  fainer 

An  important  strategic  topic  confronting  the  United  States 
of  America  is  sustaining  Department  of  Defense  (DoD) 
weapon  systems  as  part  of  the  overall  defense  life  cycle 
management  process.  For  the  past  several  decades,  billions 
of  taxpayer  dollars  have  been  spent  on  weapon  systems 
annually.  The  lives  of  U.S.  Armed  Forces  members  and 
the  people  they  protect  depend  upon  the  quality  of  these 
weapon  systems.  As  these  weapon  systems  have  become 
more  sophisticated  and  more  complex  coupled  with  a 
decrease  in  the  size  of  the  U.S.  Armed  Forces  over  the 
past  30  years,  the  military  has  become  increasingly  reliant 
on  these  weapon  systems  for  our  nation’s  security. 
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Welcome  to  the  Defense  Acquisition  Review  Journal  (ARJ)  Issue  48.  The  first 
article  in  this  issue,  “Does  Organizational  Level  Influence  Self-leadership  in  the 
Defense  Acquisition  Workforce?”  by  Trudy  C.  DiLiello  and  Jeffery  D.  Houghton  deals 
with  leadership  and  creativity  in  the  defense  acquisition  workforce,  and  summarizes  the 
findings  of  their  research.  The  authors  conducted  a  study  investigating  the  relationships 
between  self-leadership  and  creativity  in  the  context  of  a  defense  acquisition  organiza¬ 
tion.  More  specifically,  this  study  examined  differences  in  self-leadership,  creativity, 
and  perceived  organizational  support  for  creativity  between  line-  and  supervisory-level 
defense  acquisition  employees. 

The  following  article,  “Lessons  from  the  Development  of  Army  Systems,”  by  Rich¬ 
ard  G.  “Dick”  Rhoades  and  William  A.  “Bill”  Lucas  investigates  the  impact  of  program 
stability  on  program  outcomes.  Uncertainty  of  a  project’s  future  and  funding  cutbacks 
were  found  to  have  a  strong  predictive  influence  on  development  program  effectiveness. 
A  central  conclusion  from  this  study  is  that  shorter  development  cycle  times  favorably 
correlate  with  lower  levels  of  these  sources  of  program  instability,  and  with  substan¬ 
tially  better  project  outcomes. 

In  the  third  article,  “How  to  Make  Incentive  and  Award  Fees  Work,”  by  Alan  S. 
Gilbreth  and  Sylvester  Hubbard,  the  authors  conducted  a  research  effort  to  better 
understand  where  incentive  and  award  fees  had  favorable  impact  on  performance  out¬ 
comes  and  why.  This  article  summarizes  the  findings  of  the  research,  highlights  several 
organizations  that  clearly  used  techniques  that  drove  favorable  outcomes,  and  provides 
recommendations  and  take-aways  that  will  promote  effective  and  efficient  incentive  and 
award  fee  programs. 

In  the  next  article,  “PPBE:  A  Red  or  Blue  Pill?  Can  Defense  Sensemakers  Really 
be  Rational  in  a  Hyperturbulent  World?”  by  COL  Christopher  R.  Paparone,  USA  (Ret.), 
the  author  applies  social  construction  theory  to  reveal  potential  blind  spots  associated 
with  the  technical  rationality  paradigm  rooted  in  the  Defense  Department’s  Planning, 
Programming,  Budgeting  and  Execution  (PPBE)  process.  This  article  expands  upon 
Karl  E.  Weick's  version  of  sensemaking  (i.e.,  using,  modifying,  rejecting,  and  creating 


new  paradigms  or  shared  mental  models  when  dealing  with  situations  of  incoherency 
and  disorderliness)  to  examine  the  effectiveness  of  PPBE  in  a  turbulent  world. 

In  the  fifth  article,  “System  of  Systems  Development  for  the  DoD:  Tailoring  Ac¬ 
quisition  Reform  for  Emerging  Needs,”  CDR  Scott  Moran,  USN,  discusses  the  chal¬ 
lenges  of  developing  Systems  of  Systems.  When  the  Defense  Acquisition  Performance 
Assessment  (DAPA)  panel  proposed  sweeping  reforms  to  address  long-standing  prob¬ 
lems  in  defense  acquisition,  their  recommendations  did  not  anticipate  critical  challenges 
expected  in  the  development  of  an  SoS.  Defense  leaders  counting  on  revolutionary  SoS 
capabilities  must  appreciate  that  current  and  proposed  acquisition  systems  insufficiently 
facilitate  SoS  development.  This  article  describes  the  importance  of  adapting  defense 
acquisition  processes  to  enable  effective  SoS  development  and  concludes  with  proposed 
modifications  to  the  DAPA  Report  recommendations.  Tailoring  defense  acquisition 
organization,  budgeting,  and  requirements  generation  systems  to  overcome  the  chal¬ 
lenges  of  SoS  acquisition  will  be  essential  for  tomorrow’s  military  systems  to  realize 
their  potential. 

In  the  final  article,  “A  Glimpse  into  DoD  Weapon  Systems  Programs,”  by  Andy 
Fainer,  the  author  provides  a  general  overview  of  sustaining  Department  of  Defense 
(DoD)  weapon  systems  as  part  of  the  overall  defense  life  cycle  management  process. 
For  the  past  several  decades,  billions  of  taxpayer  dollars  have  been  spent  on  weapon 
systems  annually.  The  lives  of  U.S.  Armed  Forces  servicemembers  and  the  people  they 
protect  depend  upon  the  quality  and  sustainment  of  these  weapon  systems.  This  article 
integrates  several  major  themes  such  as  the  2006  Quadrennial  Defense  Review  (QDR), 
the  2005  National  Defense  Strategy,  and  logistics  transformation  (including  Future  Fo- 
gistics  Enterprise).  The  author  also  summarizes  six  Government  Accountability  Office 
(GAO)  reports  in  his  analysis. 


Dr.  Paul  Alfieri 
Executive  Editor 
Defense  ARJ 
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DOES  ORGANIZATIONAL 
LEVEL  INFLUENCE 
SELF-LEADERSHIP  IN  THE 
DEFENSE  ACQUISITION 
WORKFORCE? 

Trudy  C.  DiLiello  and  Jeffery  D.  Houghton 


This  article  presents  the  findings  of  a  study  that  investigated  the 
relationships  between  self-leadership  and  creativity  in  the  context  of  a 
defense  acquisition  organization  and  its  employees.  More  specifically,  this 
study  examined  differences  in  self-leadership,  creativity,  and  perceived 
organizational  support  for  creativity  between  line-  and  supervisory- 
level  defense  acquisition  employees.  The  resultant  analyses  suggested 
that  self-leadership  was  significantly  related  to  creative  potential  and 
practiced  creativity  for  both  line-  and  supervisory-level  employees, 
although  there  were  no  significant  differences  in  overall  levels  of  self¬ 
leadership  between  the  two  groups.  In  addition,  the  study  findings 
revealed  significant  differences  in  creative  potential,  practiced  creativity, 
gap  scores,  and  perceptions  of  organizational  support  for  creativity. 

This  article  presents  the  findings  of  a  study  that  addresses  the  question  of  whether 
there  are  significant  and  meaningful  differences  in  self-leadership  and  creativity 
between  line-  and  supervisory-level  defense  acquisition  employees.  Our  findings 
imply  that  self-leadership  is  a  primary  tool  for  facilitating  creativity  at  all  organiza¬ 
tional  levels  and  that  active  organizational  support  for  creativity  may  be  the  key  for 
reducing  the  gap  between  creative  potential  and  practiced  creativity  that  represents 
untapped  creative  resources.  Our  results  suggest  that  this  gap  is  much  more  pro¬ 
nounced  among  line  employees,  and  that  line  employees  generally  perceive  less  orga¬ 
nizational  support  for  using  their  creative  resources  than  supervisors.  In  response,  we 
make  some  specific  suggestions  for  organizational  interventions  designed  to  increase 
self-leadership  capabilities  at  all  levels  and  to  increase  perceptions  of  organization 
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support  for  creative  practices  among  line  employees  in  defense  acquisition.  Our  sug¬ 
gestions,  when  applied  across  the  defense  acquisition  workforce,  will  accelerate  the 
pace  at  which  the  Department  of  Defense  (DoD)  continues  to  develop  a  workforce  of 
creative  self-leaders,  capable  of  synergistically  assisting  organizations  in  maximizing 
the  leveraging  of  all  organizational  resources. 


Self-leadership  is  a  tontept  that  fotuses  on  self-reflettion 
and  -evaluation  aimed  at  improving  personal  and 
professional  performante. 


BACKGROUND 

Innovation  and  creativity  are  critical  for  organizations  to  thrive  in  the  21st  century 
(Kanter,  1983;  Tushman  &  O’Reilly,  1997;  Utterback,  1994).  Indeed,  the  Business 
Transformation  Agency  (BTA)  under  the  Office  of  the  Secretary  of  Defense  (OSD) 
has  placed  the  leveraging  of  innovation  and  creativity  among  the  most  effective  ap¬ 
proaches  for  creating  the  transformational  changes  needed  to  maintain  Department 
of  Defense  strategic  superiority.  Creativity  is  more  likely  to  occur  if  an  individual  has 
certain  characteristics  or  innate  skills  and  abilities  (Tierney  &  Farmer,  2002;  Hinton, 
1970;  Simonton,  1992;  Woodman  &  Schoenfeldt,  1989)  and  when  the  individual 
perceives  that  the  work  environment  supports  creativity  (Amabile,  1996;  Cummings, 
Hinton,  &  Gobdel,  1975;  Woodman,  et  al.,  1993).  Furthermore,  the  ability  to  leverage 
creativity  depends  largely  on  effective  leadership  (Kouzes  &  Posner,  1995;  Manz  & 
Sims,  2001).  A  common  theme  in  improving  leadership  effectiveness  concerns  know¬ 
ing  and  leading  oneself  (Bennis,  1994;  Drucker,  1999;  Goleman,  Boyatzis,  &  McKee, 
2002;  Senge,  1990;  Yukl,  2002).  Self-leadership  is  a  concept  that  focuses  on  self -re¬ 
flection  and  -evaluation  aimed  at  improving  personal  and  professional  performance. 

Although  theorists  have  often  suggested  relationships  between  self-leadership 
and  creativity  (e.g.,  Kouzes  &  Posner,  1995;  Manz  &  Sims,  2001),  very  little  atten¬ 
tion  has  been  given  to  how  these  relationships  may  differ  across  an  organizational 
level.  The  purpose  of  the  current  study  is  to  address  the  question  of  whether  there 
are  significant  and  meaningful  differences  in  self-leadership  and  creativity  between 
line-  and  supervisory-level  defense  acquisition  employees.  The  answer  to  this  ques¬ 
tion  may  have  important  implications  for  maximizing  employee  self-direction  and  for 
fully  leveraging  creative  resources  at  all  organizational  levels. 
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CREATIVE  POTENTIAL  AND  PRACTICED  CREATIVITY 


Although  creativity  is  a  complex  concept  that  is  somewhat  difficult  to  define, 
consistent  themes  tend  to  emerge  across  the  various  definitions  in  the  creativity  litera¬ 
ture  (e.g.,  Barron  &  Harrington,  1981;  Guilford,  1950;  Martindale,  1989;  Sternberg 
and  Lubart,  1999).  Based  on  the  common  ideas  in  these  definitions,  we  define  creativ¬ 
ity  as  an  ability  to  harvest  novel  but  appropriate  ideas  in  order  to  maximize  efficien¬ 
cies,  solve  problems,  and  increase  effectiveness.  We  further  divide  the  creativity 
concept  into  creative  potential  and  practiced  creativity  (e.g.,  Hinton,  1968;  DiLiello 
&  Houghton,  2006,  2008).  In  short,  if  an  individual’s  creativity  is  attenuated  by  the 
environment,  then  the  individual  will  not  use  his  or  her  full  creative  potential  (Hinton, 
1968;  George  &  Zhou,  2001;  Scott,  1965). 

Creative  potential  is  the  creative  capacity,  skills,  and  abilities  that  a  person 
possesses  (Hinton,  1968,  1970).  Creative  potential  includes  the  concept  of  creative 
self-efficacy,  an  individual’s  subjective  assessment  of  his  or  her  personal  ability  to 
be  creative  (Tierney  and  Farmer,  2002).  Creative  self-efficacy  involves  seeing  one¬ 
self  as  being  good  at  creative  problem  solving  and  generating  novel  ideas.  Creative 
potential  also  includes  having  the  talent  or  expertise  to  do  well  in  one’s  work  and 
possessing  the  ability  to  take  risks  by  trying  out  new  ideas  (Amabile,  Burnside,  & 
Gryskiewicz,  1999). 


The  gap  between  freative  potential  and  creative  practice 
represents  untapped  organizational  resources. 


Practiced  creativity,  on  the  other  hand,  is  the  perceived  opportunity  to  use  cre¬ 
ativity  skills  and  abilities.  Practiced  creativity  should  not  be  confused  with  creative 
performance,  which  is  an  external  assessment  of  products  or  achievements  (Amabile, 
1996;  Hinton,  1968).  Employees  with  strong  creative  potential  are  more  likely  to 
actually  practice  creativity  when  they  perceive  strong  support  from  the  organization 
(DiLiello  &  Houghton,  2006),  and  several  key  conditions  must  be  present  within  an 
organization  for  its  work  environment  to  support  individual  creativity  (e.g.,  Amabile, 
1988;  Ford,  1996;  Mumford  &  Gustafson,  1988). 

The  distinction  between  creative  potential  and  practiced  creativity  is  important 
because  when  people  perceive  themselves  as  having  creative  potential  but  do  not 
perceive  the  ability  to  use  or  practice  this  potential,  they  will  be  less  likely  to  engage 
in  creative  behavior.  The  gap  between  creative  potential  and  creative  practice  repre¬ 
sents  untapped  organizational  resources.  Identifying  such  untapped  resources  may  be 
especially  important  in  defense  acquisition  organizations  that  are  continually  being 
told  to  “do  more  with  less.” 
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PERCEIVED  ORGANIZATIONAL  SUPPORT  FOR  CREATIVITY 


Over  the  past  two  decades,  the  general  concept  of  perceived  organizational  sup¬ 
port  (POS)  has  become  a  central  organizational  construct  that  has  been  investigated  in 
a  number  of  empirical  studies  (Rhoades  &  Eisenberger,  2002).  Based  on  the  concept 
of  social  exchange  (e.g.,  Eisenberger  Huntington,  Hutchison,  &  Sowa,  1986)  and 
the  norm  of  reciprocity  (Gouldner,  1960),  POS  theory  suggests  that  employees  will 
demonstrate  commitment,  enhanced  performance,  and  decreased  withdrawal  behav¬ 
iors  in  exchange  for  fair  procedures,  support  from  their  supervisor,  desirable  rewards, 
and  favorable  job  conditions  (Rhoades  &  Eisenberger,  2002).  Research  studies  have 
shown  empirical  linkages  between  POS  and  job  conscientiousness,  job  involvement, 
loyalty,  trust  in  the  organization,  and  decreased  turnover  (Eisenberger,  Fasolo,  & 
Davis-LaMastro,  1990;  Eisenberger,  Huntington,  Hutchison,  &  Sowa,  1986). 

Research  has  also  suggested  a  relationship  between  POS  and  innovation/creativ¬ 
ity  (Amabile,  1988;  Cummings,  Hinton  &  Gobdel,  1975,  Eisenberger,  et  al.,  1990; 
Scott  &  Bruce,  1994;  Shalley,  1995;  Woodman,  Sawyer  &  Griffen,  1993;  Zhou  & 
George,  2001).  Indeed,  some  researchers  have  advanced  the  concept  of  perceived 
organizational  support  for  creativity  as  a  specific  type  of  POS  in  organizations 
(e.g.,  Zhou  &  George,  2001).  Perceived  organizational  support  for  creativity  can  be 
defined  as  “the  extent  to  which  an  employee  perceives  that  the  organization  encour¬ 
ages,  respects,  rewards,  and  recognizes  employees  who  exhibit  creativity”  (Zhou  & 
George,  2001,  p.  686).  Perceived  organizational  support  for  creativity  can  be  further 
conceptualized  in  terms  of  “an  organizational  culture  that  encourages  creativity 
through  the  fair,  constructive  judgment  of  ideas,  reward  and  recognition  for  creative 
work,  mechanisms  for  developing  new  ideas  and  active  flow  of  ideas,  and  a  shared 
vision  of  what  the  organization  is  trying  to  do”  (Amabile  et  ah,  1999,  p.  15).  Per¬ 
ceived  organizational  support  for  creativity  is  conceptually  distinct  from  practiced 
creativity,  which  focuses  more  on  actual  opportunities  to  use  creative  skills  rather 
than  upon  an  environment  that  rewards  and  encourages  creative  behavior  (DiLiello 
&  Houghton,  2008). 

SELF-LEADERSHIP 

Self-leadership  (e.g.,  Manz,  1986;  Neck  &  Houghton,  2006;  Neck  &  Manz, 

2007)  is  a  self-evaluation  and  self-influence  process  through  which  individuals 
identify  and  replace  ineffective  behaviors  and  negative  thought  processes  with  more 
effective  behaviors  and  positive  thought  processes,  thereby  enhancing  personal  ac¬ 
countability  and  improving  professional  performance.  Theorists  have  long  suggested 
that  leaders  in  organizations  should  encourage  their  followers  to  lead  themselves  in 
the  workplace  (e.g.,  Manz  &  Sims,  1980,  2001).  Supervisors  and  work  environments 
only  have  a  limited  control  over  the  workers;  additional  control  or  work  motivation 
must  come  from  within  the  individual  (Herzberg,  Mausner,  &  Snyderman,  2003; 
Manz  &  Sims,  1980;  Sergiovanni,  1992).  When  employees  are  trained  and  empow¬ 
ered  to  lead  themselves,  supervisors  can  shift  their  focus  from  detailed  oversight  and 
control  to  longer-term  big  picture  issues. 
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Founded  upon  several  classic  theories  of  self-influence  including  self-regulation 
(Kanfer,  1970;  Carver  &  Scheier,  1981),  self-control  (Cautela,  1969;  Mahoney  & 
Arnkoff,  1978,  1979;  Thoresen  &  Mahoney,  1974),  intrinsic  motivation  theory  (e.g., 
Deci  and  Ryan,  1985),  and  social  cognitive  theory  (e.g.,  Bandura,  1986),  self-leader¬ 
ship  is  a  normative  model  that  prescribes  specific  sets  of  behavioral  and  cognitive 
strategies  aimed  at  increasing  individual  performance.  Self-leadership  strategies  are 
often  divided  into  three  primary  categories:  Behavior  Focused  Strategies,  Natural  Re¬ 
ward  Strategies,  and  Constructive  Thought  Strategies  (e.g.,  Neck  &  Houghton,  2006). 


When  employees  are  trained  and  empowered  to  lead 
themselves ,  supervisors  fan  shift  their  fotus  from  detailed 
oversight  and  tontrol  to  longer-term  big  pitture  issues . 


Behavior  Focused  Strategies.  This  category  involves  identifying  and  replacing 
ineffective  behaviors  with  more  effective  ones  through  a  process  of  self-observation, 
self-goal  setting,  self -reward,  and  self-correcting  feedback  (Neck  &  Houghton,  2006). 
Self-observation  entails  a  close  examination  of  one’s  own  behaviors  in  order  to  iden¬ 
tify  behaviors  that  should  be  changed,  enhanced,  or  eliminated  (Mahoney  &  Arnkoff, 
1978,  1979;  Manz  &  Sims,  1980;  Neck  &  Manz,  2007).  Once  target  behaviors  have 
been  identified,  individuals  can  establish  goal  and  associated  reward  contingencies  to 
energize  and  direct  necessary  behaviors  (Mahoney  &  Arnkoff,  1978,  1979;  Manz  & 
Sims,  1980;  Neck  &  Manz,  2007).  Additionally,  self-correcting  feedback,  consisting 
of  a  positively  framed  reflection  on  failures  and  undesirable  behaviors,  may  be  quite 
effective  in  helping  to  recast  these  behaviors  in  more  positive  directions  (Manz  & 
Sims,  2001). 

Natural  Reward  Strategies.  This  category  includes  the  ability  of  the  individual 
to  find  pleasure  in  the  work  that  has  to  be  performed  and  to  focus  on  the  inherently 
enjoyable  aspects  of  task  or  activity,  leading  to  increased  feelings  of  competence, 
self-control,  and  a  sense  of  purpose  (Csikszentmihalyi,  1996;  Deci  &  Ryan,  1985; 
Herzberg  et  al,  2003).  Natural  reward  strategies  include  building  more  pleasant  and 
enjoyable  features  into  a  task  or  activity  so  that  the  task  itself  becomes  more  intrinsi¬ 
cally  rewarding,  and  shifting  mental  focus  to  inherently  rewarding  aspects  of  the  task 
(Neck  &  Houghton,  2006;  Neck  &  Manz,  2007). 

Constructive  Thought  Strategies.  This  category  focuses  on  directing  and  reshap¬ 
ing  various  mental  processes  including  beliefs  and  assumptions,  self-verbalizations 
(self-talk),  and  mental  imagery  in  order  to  create  constructive  thought  patterns  and 
habitual  ways  of  thinking  that  may  have  a  positive  impact  on  individual  performance 
(Neck  &  Houghton,  2006;  Neck  &  Manz,  1992,  1996).  For  example,  individuals 
can  assess  their  thought  patterns  in  an  effort  to  identify  and  eliminate  dysfunctional 
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beliefs  and  assumptions  with  more  rational  and  constructive  ones  (Burns,  1980;  Ellis, 
1977;  Neck  &  Manz,  1992).  Similarly,  self-talk,  defined  as  what  we  covertly  tell 
ourselves,  can  be  closely  examined  in  order  to  eliminate  undue  negativity  and  pes¬ 
simism.  Research  in  various  fields  (sports  psychology,  clinical  psychology,  educa¬ 
tion,  and  communication)  supports  the  use  of  positive  self-talk  as  an  effective  way  to 
improve  individual  performance  (e.g.,  Neck  &  Manz,  1992).  Mental  imagery  involves 
symbolically  experiencing  behavioral  outcomes  prior  to  actual  performance  without 
overt  physical  muscular  movement  (Driskell,  Copper,  &  Moran,  1994;  Finke,  1989; 
Neck  &  Manz,  1992,  1996).  Research  suggests  that  people  who  visualize  success¬ 
ful  performance  before  actually  engaging  in  performance  are  much  more  likely  to 
perform  successfully  when  faced  with  the  actual  task  (Neck  &  Houghton,  2006).  In  a 
meta-analysis  of  35  empirical  studies,  Driskell  et  al.  (1994)  reported  an  overall  posi¬ 
tive  and  significant  effect  for  mental  imagery  on  individual  performance. 


The  relationship  between  rreativity  and  self-leadership  may 
be  partially  founded  on  the  tontepts  of  autonomy  and 
self-determination . 


Theorists  have  often  suggested  a  relationship  between  self-leadership  and  creativ¬ 
ity  (e.g.,  DiLiello  &  Houghton,  2006;  Houghton  &  Yoho,  2005;  Manz  &  Sims,  2001). 
The  relationship  between  creativity  and  self-leadership  may  be  partially  founded  on 
the  concepts  of  autonomy  and  self-determination.  Autonomy,  a  key  aspect  of  creativ¬ 
ity  (e.g.,  Amabile,  1996;  Barron  &  Harrington,  1981;  Woodman  et  al.,  1993),  has 
been  linked  to  self-determination  and  intrinsic  motivation  (Deci  &  Ryan,  1985).  Self- 
determination  is  a  primary  component  of  self-leadership’s  natural  reward  strategies 
(Neck  &  Manz,  2007).  Indeed,  empirical  research  suggests  that  an  individual’s  need 
for  autonomy  can  subsequently  influence  the  extent  to  which  the  individual  engages 
in  self-leadership  (Yun,  Cox,  &  Sims,  2006). 

Other  relationships  between  creativity  and  self-leadership  have  also  been  suggest¬ 
ed.  For  example,  Houghton  and  Yoho  (2005)  have  suggested  a  relationship  between 
individual  self-leadership  and  subsequent  levels  of  individual  independence  and 
creativity.  In  addition,  internal  locus  of  control,  a  theorized  component  of  creativity, 
has  been  empirically  related  to  individual  self-leadership  (Kazan  &  Earnest,  2000). 
Finally,  an  empowering  leadership  style  (leading  others  to  be  self-leaders)  tends  to 
promote  creativity  rather  than  conformity  (Manz  &  Sims,  2001).  Indeed,  creativ¬ 
ity  may  be  one  of  the  most  essential  aspects  of  effective  organizational  leadership 
(Mumford  &  Connelly,  1999).  Creative  thinking  and  a  different  style  of  leadership 
are  necessary  to  provide  flexibility,  facilitate  change,  and  redesign  traditional  bureau¬ 
cratic  processes  (Katz  &  Kahn,  1978).  Encouraging  self-leadership  is  a  relatively 
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new  leadership  style  that  may  help  to  promote  an  organizational  climate  that  supports 
creativity.  Empowering  leadership  is  rapidly  becoming  a  key  success  strategy  in  the 
rapidly  changing  work  environments  of  the  21st  century. 


PURPOSE  AND  RATIONALE 

The  purpose  of  the  current  study  is  to  examine  the  relationships  between  self¬ 
leadership  and  creativity  in  the  context  of  a  defense  acquisition  organization.  Our 
primary  research  question  is:  Are  there  significant  differences  in  self-leadership  and 
creativity  between  line-  and  supervisory-level  defense  acquisition  employees?  More 
specifically,  our  analysis  will  address  the  following  questions:  1)  Are  there  significant 
differences  in  self-leadership  between  line-  and  supervisory-level  employees?  2)  Are 
there  significant  differences  in  creative  potential,  practiced  creativity,  and  the  gap 
between  the  two  in  line-  and  supervisory-level  employees?  3)  Are  there  significant 
differences  in  perceived  organizational  support  for  creativity  between  line-  and  super¬ 
visory-level  employees? 

The  present  study  contributes  to  the  self-leadership  and  creativity  literature  in 
a  number  of  important  ways.  First,  this  study  takes  an  empirical  step  toward  un¬ 
derstanding  the  nature  of  the  relationship  between  self-leadership  and  creativity. 

This  study  also  examines  the  role  of  organizational  support  in  facilitating  practiced 
creativity  among  organizational  members.  Most  importantly,  this  study  is  among  the 
first  to  examine  differences  in  self-leadership,  creativity,  and  perceptions  of  support 
between  line-  and  supervisory-level  employees.  Understanding  these  differences  may 
be  critical  for  reducing  the  gap  between  creative  potential  and  practiced  creativity  in 
organizations.  Finally,  this  study  makes  a  unique  contribution  to  our  knowledge  of 
creativity  and  self-leadership  in  the  context  of  defense  acquisition.  The  differences 
examined  here  may  have  important  implications  for  creating  a  defense  acquisition 
workforce  with  strong  self-leaders  working  in  environments  that  support  creativity. 
Creative  self-leaders  could  synergistically  assist  the  DoD  in  maintaining  an  all-im¬ 
portant  competitive  advantage  in  the  face  of  a  wide  range  of  21st  century  challenges. 


METHOD 


SAMPLE  AND  PROCEDURE 

Primary  data  were  collected  from  the  Army  Contracting  Agency  (ACA)  as  part  of 
a  larger  study  that  examined  a  number  of  performance-related  issues.  Approximately 
37  percent  of  the  total  ACA  workforce  of  approximately  1 ,900  people  chose  to  com¬ 
plete  the  online  survey — a  fairly  high  response  rate  when  compared  to  the  response 
rates  for  other  federal  employee  surveys  and  with  response  rates  for  e-mail  surveys  in 
general  (Sheehan,  2001).  Fist- wise,  deletion  for  missing  data  resulted  in  a  final  over¬ 
all  sample  of  654.  This  sample  was  subsequently  divided  into  two  subsamples  (i.e., 
supervisory  employees,  N=2 1 5;  and  line  employees,  N=439)  for  further  analysis.  The 
average  age  of  the  respondents  was  approximately  46,  and  the  average  job  tenure  was 
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approximately  12  years.  Sixty  percent  of  the  respondents  were  female.  The  online 
survey  was  activated  in  accordance  with  the  tailored  design  method  (Dillman,  2000). 
An  initial  e-mail  was  sent  to  ACA  workforce  members  that  included  an  Informed 
Consent  Notification,  the  purpose  of  the  study,  the  approval  and  sponsorship  of  the 
study,  a  confidentiality  statement,  and  a  link  to  the  online  survey.  A  subsequent  fol¬ 
low-up  e-mail  summarized  the  first  message,  added  a  personal  note,  and  provided  a 
four-day  extension  along  with  a  link  to  the  online  survey. 

MEASURES 

Self -leadership.  Thirteen  items  from  the  Revised  Self-leadership  Questionnaire 
(RSLQ,  Houghton  &  Neck,  2002)  were  used  to  measure  self-leadership.  The  RSLQ 
has  been  used  to  measure  self-leadership  in  numerous  studies  (e.g.,  Houghton  &  Jink- 
erson,  2007;  Houghton,  Bonham,  Neck  &  Singh,  2004;  Neubert  &  Wu,  2006).  The  13 
items  demonstrated  good  reliability  in  the  current  sample  with  an  alpha  coefficient  of 
.80 — well  above  Nunnally’s  (1978)  recommended  threshold  of  .70.  The  items  were 
measured  using  a  five-point  Likert-type  scale  ranging  from  Completely  Accurate  to 
Not  At  All  Accurate. 

Creative  potential  and  practiced  creativity.  Eleven  items  were  used  to  measure 
creativity,  with  six  items  assessing  creative  potential  and  five  items  representing 
practiced  creativity  (DiLiello  &  Houghton,  2008).  DiLiello  and  Houghton  (2008) 
assessed  the  construct  and  discriminant  validity  of  the  creative  potential  and  prac¬ 
ticed  creativity  concepts  using  an  exploratory  factor  analysis  (EFA)  along  with  a 
confirmatory  factor  analysis  (CFA)  using  structural  equation  modeling  techniques. 
Their  EFA  demonstrated  a  clean  factor  structure  for  each  construct  with  strong  factor 
loadings  and  virtually  no  cross-loadings  for  any  of  the  items,  while  the  reliability  of 
the  items  used  to  measure  each  construct  was  also  quite  good,  with  coefficient  alphas 
of  .84  for  each  of  the  two  constructs  (DiLiello  &  Houghton,  2008).  CFA  results  also 
provided  additional  evidence  in  support  of  the  construct  and  discriminant  validity  of 
these  concepts  (DiLiello  &  Houghton,  2008).  Items  were  measured  using  a  five -point 
Likert-type  scale  ranging  from  Strongly  Agree  to  Strongly  Disagree. 

Perceived  organizational  support  for  creativity.  Perceived  organizational  support 
for  creativity  was  measured  with  six  items  from  “KEYS:  Assessing  the  Climate  for 
Creativity,”  used  with  the  permission  of  the  Center  for  Creative  Leadership  (Amabile 
et  al.,  1999).  The  KEYS  scale  has  shown  good  psychometric  properties  as  evidenced 
by  CFA  results  and  by  median  reliability  estimates  of  .84  across  a  number  of  studies 
(e.g.,  Mathisen  &  Einarsen,  2004).  All  items  were  measured  using  a  five -point  Likert- 
type  scale  ranging  from  Strongly  Agree  to  Strongly  Disagree. 

ANALYSES 

Mean  differences  between  supervisory  and  line  employees  for  self-leadership, 
creative  potential,  practiced  creativity,  a  gap  score  (i.e.,  the  difference  between 
creative  potential  and  practiced  creativity  that  represents  untapped  creative  potential), 
and  perceptions  of  organizational  support  for  creativity  were  examined  using  a  series 
of  t-tests.  In  addition,  a  series  of  regression  analyses  were  conducted  to  examine  the 
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effects  of  self-leadership,  perceived  organizational  support  for  creativity  and  organi¬ 
zational  level  (line  vs.  supervisory)  on  creative  potential,  practiced  creativity,  and  gap 
scores,  respectively,  along  with  the  effects  of  organizational  level  (line  vs.  supervi¬ 
sory)  on  perceived  organizational  support  for  creativity. 


RESULTS 

Means  and  standard  deviations  for  both  supervisory  and  line  employees  for 
self-leadership,  creative  potential,  practiced  creativity,  gap  scores,  and  perceived 
organizational  support  for  creativity  are  shown  in  Table  1 .  The  analysis  indicated 
no  mean  difference  between  groups  for  self-leadership,  t(507df)  =  1.16,  p  =  .247.  In 
contrast,  analyses  showed  significant  mean  differences  between  the  two  groups  for 
creative  potential,  t(652df)  =  3.30,  p  =  .001;  practiced  creativity,  t(469df)  =  7.48,  p  = 
.000;  gap  scores,  t(471df)  =  -5.03,  p  =  .000;  and  perceived  organizational  support  for 
creativity,  t(652df)  =  3.21,  p  =  .001. 

Four  separate  regression  analyses  were  conducted.  Model  1  examined  the  effects 
of  the  independent  variables  self-leadership  and  organizational  level  ( 1  =supervisor 
-  0=line,  using  dummy  variable  coding)  on  the  dependent  variable  creative  potential. 
Model  2  examined  the  effects  of  self-leadership,  perceived  organizational  support 
for  creativity,  and  organizational  level  on  the  dependent  variable  practiced  creativity. 
Model  3  examined  the  relationships  between  the  three  independent  variables  and  gap 
scores.  Finally,  Model  4  explored  the  effects  of  organizational  level  on  perceptions 
of  organizational  support  for  creativity.  A  summary  of  the  results  of  these  analyses  is 
presented  in  Table  2. 

The  regression  equation  for  Model  1  suggested  that  both  self-leadership  and 
organizational  level  were  significantly  related  to  creative  potential,  with  self-leader- 
ship  as  the  stronger  predictor  of  the  two  (Standardized  13  =  .356,  p  =  .000).  The  equa¬ 
tion  for  Model  2  indicated  that  self-leadership,  perceived  organizational  support  for 


TABLE  1.  MEANS  AND  STANDARD  DEVIATIONS  (IN  PARENTHESES) 


SL 

CP 

PC 

GS 

OS 

Supervisors 

49.55 

25.47 

23.58 

1.89 

20.00 

N=215 

(6.10) 

(2.98) 

(4.04) 

(4.10) 

(5.40) 

Line  Employees 

48.92 

24.65 

20.97 

3.68 

18.54 

N=439 

(7.43) 

(3.03) 

(4.51) 

(4.60) 

(5.46) 

Note.  SL=Self-Leadership,  CP=Creative  Potential,  PC=Practiced  Creativity, 
GS=Gap  Score,  OS=Perceived  Organizational  Support. 
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creativity,  and  organizational  level  were  all  significant  predictors  of  practiced  creativ¬ 
ity,  accounting  for  approximately  42.6  percent  of  its  variance.  Of  the  three  variables, 
perceived  organizational  support  was  the  stronger  predictor  of  practiced  creativity 
(Standardized  13  =  .563,  p  =  .000).  The  Model  3  analysis  found  that  perceived  orga¬ 
nizational  support  and  organizational  level  were  significantly  and  negatively  related 
to  gap  scores,  explaining  approximately  33.1  percent  of  the  observed  variance.  The 
regression  equation  suggested  a  strong  negative  effect  for  perceived  organizational 
support  (Standardized  13  =  -.551,  p  =  .000),  indicating  that  lower  perceptions  of 
organizational  support  for  creativity  will  result  in  larger  gaps  between  an  individual’s 
creative  potential  and  their  practiced  creativity.  In  addition,  the  equation  suggests 
that  gap  scores  will  be  significantly  greater  for  line  employees  than  for  supervisors 
(Organizational  Level:  Standardized  13  =  -.1 17,  p  =  .000).  Finally,  the  regression 
analysis  for  Model  4  implied  that  supervisors  tend  to  have  more  positive  perceptions 
of  organizational  support  for  creativity  than  line  employees  (Organizational  Level: 
Standardized  13  =  .125,  p  =  .001). 


TABLE  2.  SUMMARY  OF  REGRESSION  ANALYSES  RESULTS 


Independent 

Variables 

Model  1: 

3 

Creative  Potential 
p-  value 

Model  2: 

3 

Practiced  Creativity 
p-  value 

Model  3: 

1. 3 

Gap  Score 
p-  value 

Model  4: 

3 

Organizational  Support 
p-  value 

Self-Leader¬ 

ship 

.356 

.000 

.158 

.000 

Perceived 

Organizational 

Support 

.563 

.000 

-.551 

.000 

Organizational 

Level 

.113 

.002 

.195 

.000 

-.117 

.000 

.125 

.001 

Adjusted  R2 

.140 

.426 

.331 

.014 

F  Statistic 

54.25 

162.84 

162.53 

10.32 

p  -  value 

.000 

.000 

.000 

.001 
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DISCUSSION 


This  study  revealed  a  number  of  significant  differences  between  line  and  super¬ 
visory  defense  acquisition  employees.  Our  analyses  suggested  that  self-leadership 
was  significantly  related  to  creative  potential  and  practiced  creativity  for  both  line- 
and  supervisory-level  employees  with  no  significant  differences  in  overall  levels  of 
self-leadership  between  the  two  groups.  In  contrast,  we  found  significant  differences 
between  line-  and  supervisory-level  employees  in  creative  potential,  practiced  creativ¬ 
ity,  gap  scores,  and  perceptions  of  organizational  support  for  creativity.  Specifically, 
line  employees  reported  significantly  lower  levels  of  creative  potential,  practiced 
creativity,  and  perceptions  of  organizational  support  for  creativity  along  with  higher 
gap  scores  in  comparison  to  supervisors. 


Self-leadership  appears  to  be  the  more  important  iomept  in 
determining  an  individual's  creative  potential . 


Our  analyses  further  suggested  that  although  supervisors  tend  to  have  more  cre¬ 
ative  potential  than  line  employees,  self-leadership  appears  to  be  the  more  important 
concept  in  determining  an  individual’s  creative  potential.  Likewise,  although  self¬ 
leadership  and  organizational  level  are  both  important  determinants  of  practiced  cre¬ 
ativity,  employee  perceptions  of  organizational  support  for  creativity  seem  to  be  far 
more  crucial.  Similarly,  perceived  organizational  support  for  creativity  appears  to  be 
more  important  than  organizational  level  in  predicting  creativity  gaps  in  acquisition 
employees.  In  other  words,  employees  who  feel  that  the  organization  supports  their 
creative  efforts  will  be  much  more  likely  to  practice  creative  behaviors,  thus  lowering 
the  gap  between  their  potential  and  practiced  creativity.  Finally,  organizational  level 
was  a  significant  determinant  of  perceptions  of  organizational  support  for  creativ¬ 
ity,  with  supervisory  employees  holding  significantly  more  positive  perceptions  of 
support  than  line  employees.  In  summation,  our  analyses  suggest  that  self-leadership 
may  be  a  key  determinant  of  creative  potential  and  practice  among  defense  acquisi¬ 
tion  employees;  and  that  perceptions  of  organizational  support  for  creativity,  which 
tend  to  be  weaker  in  non-supervisory  employees,  are  critical  in  determining  whether 
creative  potential  will  be  realized  or  whether  a  gap  between  potential  and  practice 
will  result. 

The  results  of  this  study  have  important  theoretical,  empirical,  and  practical 
applications  that  add  to  our  understanding  of  the  nature  of  the  relationship  between 
self-leadership,  creativity,  and  organizational  support  for  creative  practices  at  both 
the  supervisory  and  non-supervisory  levels.  Our  findings  imply  that  self-leadership 
is  a  primary  tool  for  facilitating  creativity  at  all  organizational  levels  and  that  active 
organizational  support  for  creativity  may  be  the  key  for  reducing  the  gap  between 
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creative  potential  and  practiced  creativity  that  represents  untapped  creative  resources 
Our  results  also  suggest  that  this  gap  is  much  more  pronounced  among  line  employ¬ 
ees  and  that  line  employees  generally  perceive  less  organizational  support  for  using 
their  creative  resources  than  supervisors.  To  address  this  situation,  an  organizational 
intervention  designed  to  increase  self-leadership  capabilities  at  all  levels  and  to 
increase  perceptions  of  organization  support  for  creative  practices  among  line  em¬ 
ployees  in  defense  acquisition  would  be  well  advised.  More  specifically,  a  structured 
self-leadership  training  program  similar  to  those  reported  elsewhere  in  the  literature 
(e.g.,  Neck  &  Manz,  1996;  Stewart,  Carson,  &  Cardy,  1996)  could  be  conducted  for 
defense  acquisition  employees.  Such  a  training  program  could  have  the  dual  effect 
of  increasing  self-leading  behaviors  and  thus  creative  potential  while  also  strongly 
signaling  organizational  support  for  creative  behaviors. 


Employees  who  feel  that  the  organization  supports  their 
treative  efforts  will  be  much  more  likely  to  prattite 
creative  behaviors. 


Although  our  findings  suggest  exciting  avenues  toward  increasing  self-leadership 
and  unleashing  creative  resources  at  all  organizational  levels,  our  study  is  bound  by 
certain  limitations.  First,  the  present  sample  was  relatively  homogeneous,  consisting 
entirely  of  members  of  the  ACA.  As  we  have  suggested,  such  a  sample  is  especially 
appropriate  for  creativity  research  because  the  DoD  has  taken  a  keen  interest  in 
tapping  all  creative  resources  available  in  order  to  sustain  a  competitive  advantage. 
However,  whether  the  results  reported  here  would  generalize  to  other  samples  of 
interest  remains  uncertain.  Second,  all  items  were  self-reported  and  collected  using 
a  single  survey  at  a  single  point  in  time,  thus  raising  concerns  regarding  measure¬ 
ment  issues  such  as  response  set  and  social  desirability  biases.  Given  this  potential 
problem,  our  findings  should  be  viewed  with  some  degree  of  caution.  On  the  other 
hand,  despite  such  inherent  limitations,  the  use  of  self-reported  items  collected  in  a 
single  administration  is  common  practice  in  many  aspects  of  social  science  research. 
Finally,  it  is  impossible  to  determine,  based  on  these  data  and  statistical  techniques 
alone,  the  direction  of  causality  for  the  observed  relationships.  Direction  of  causality 
must  be  inferred  by  underlying  theory.  Although  we  have  advanced  empirical  and 
theoretical  arguments  supporting  the  possible  direction  of  causality  for  the  various 
relationships  reported  here,  these  arguments  cannot  be  unequivocally  substantiated  on 
the  sole  basis  of  statistical  test  results. 

Future  research  should  continue  to  examine  the  relationships  between  self-leader- 
ship,  creative  potential,  practiced  creativity,  organizational  level  and  organizational 
support  for  creativity.  Specifically,  future  research  should  more  closely  examine  the 
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role  of  organizational  support  as  a  moderator  of  the  relationship  between  creative 
potential  and  practiced  creativity  and  as  a  key  mechanism  for  reducing  the  gap 
between  these  concepts  in  organizations.  In  addition,  perceptions  of  support  for 
creativity  might  be  further  subdivided  from  the  organizational  level  to  the  work  group 
and  supervisory  levels  in  order  to  provide  additional  insights  (DiLiello  &  Houghton, 


An  acquisition  workforce  of  creative  seif -leaders  could 
synergistically  assist  the  organization  in  maximizing  the 
leveraging  of  all  organizational  resources . 


2006).  Similarly,  future  research  could  continue  to  examine  the  differences  between 
line-  and  supervisory-level  employees  in  terms  of  creativity  and  perceptions  of  sup¬ 
port  for  creative  practices,  with  an  eye  toward  identifying  ways  to  increase  creativity 
at  all  organizational  levels.  In  closing,  our  findings  and  suggestions  have  significant 
practical  application  in  the  context  of  transformational  efforts  across  the  DoD  in  sup¬ 
port  of  warfighter  readiness.  An  acquisition  workforce  of  creative  self-leaders  could 
synergistically  assist  the  organization  in  maximizing  the  leveraging  of  all  organiza¬ 
tional  resources. 
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LESSONS  FROM  THE 
DEVELOPMENT  OF 
ARMY  SYSTEMS 

William  A.  Lucas  and  Richard  G.  Rhoades 


Several  of  the  statistically  significant  relationships  found  in  this  study 
of  1  3  Army  systems  involve  factors  that  are  related  to  the  stability 
of  the  program.  For  example,  uncertainty  of  a  project's  future  and 
funding  cutbacks  are  found  to  have  a  strong  predictive  influence  on 
development  program  effectiveness,  which  may  be  explained  in  part  by 
their  impact  on  program  staffing  turnover  and  the  disruption  of  testing 
regimes.  A  central  conclusion  from  this  study  is  that  shorter  development 
cycle  times  favorably  correlate  with  lower  levels  of  these  sources  of 
program  instability,  and  with  substantially  better  project  outcomes. 

This  article  is  based  on  the  results  of  Army  Materiel  Command-sponsored  re¬ 
search  of  several  years’  duration  (Lucas  &  Rhoades,  2004).  A  structured  case 
study  approach  was  employed  to  examine  the  history  and  processes  that  had 
resulted  in  the  introduction  of  a  number  of  technology-based  Army  systems  in  time  to 
make  a  positive  contribution  to  the  outcome  of  DESERT  STORM.  The  13  case  stud¬ 
ies  that  resulted  were  developed  on  systems  ranging  from  the  M829A1  “silver  bullet” 
to  the  GUARDRAIL  Common  Sensor  and  the  APACHE  attack  helicopter. 


METHODOLOGY 

The  case  studies  were  prepared  largely  based  on  interviews  with  key  partici¬ 
pants  from  the  government/contractor  team  that  developed  each  system,  and  using  a 
research  questionnaire  to  structure  the  discussions.  The  authors  designed  the  ques¬ 
tionnaire  to  provide  coverage  of  a  number  of  development  process,  organizational 
relationship,  critical  technology  maturity,  and  other  factors  that  either  the  authors’ 
prior  experience  or  the  literature  (GAO,  1999;  GAO,  2000)  suggested  might  be 
relevant  to  determining  the  relative  success  of  projects.  A  portion  of  the  questionnaire 
consisted  of  questions  that  were  in  common  with  a  research  instrument  successfully 
used  by  one  of  the  authors  in  a  prior  study  (Air  Force  Research  Laboratory,  2002)  of 
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aerospace  research  projects.  This  process  resulted  in  collection  of  a  common  set  of 
data  for  the  systems  studied,  which  could  then  be  analyzed  to  identify  factors  contrib¬ 
uting  to  the  relative  degree  of  success  in  system  development. 


The  heart  of  any  systematic  analysis  is  the  definition  of  a 
iommon  outcome  measure  that  allows  comparison . 


The  heart  of  any  systematic  analysis  is  the  definition  of  a  common  outcome  mea¬ 
sure  that  allows  comparison.  In  this  study,  the  projects  (cases)  were  compared  based 
on  their  performance  relative  to  their  agreed-upon  goals  and  requirements.  Each 
project  had  a  budget,  a  systems  procurement  cost  goal,  a  set  of  technical  require¬ 
ments,  and  completion  dates.  In  addition,  questions  of  performance  are  immediately 
observable  and  easily  remembered  by  project  managers:  Once  production  was  started, 
were  problems  found  that  required  that  further  engineering  changes  be  made?  And 
did  the  system  perform  well  during  its  use  in  DESERT  STORM?  Using  structured 
questions,  the  researchers  asked  the  key  government  and  industry  interviewees  how 
well  their  projects  performed  in  these  areas,  with  a  range  of  answers  that  character¬ 
ized  how  badly  the  projects  had  missed  meeting  their  objectives  if  they  had  not  been 
completely  successful.  Note  that  researchers  developing  each  case  study  had  the 
independent  views  of  at  least  two  senior  managers,  as  well  as  their  own  detailed  study 
of  their  project,  to  enable  them  to  make  summary  judgments  on  project  success  meet¬ 
ing  these  largely  observable  outcomes. 

Six  of  these  outcome  measures  were  used  to  create  a  scale  that  scores  the  projects 
from  zero  to  six  according  to  the  number  of  key  outcomes  a  project  achieved.  If  a 
project  was  (a)  transitioned  to  production  on  time,  (b)  developed  within  budget,  (c) 
had  no  late  engineering  changes,  (d)  met  the  goals  for  system  unit  costs,  (e)  met  the 
goals  for  technical  requirements,  and  (f)  encountered  no  difficulties  when  deployed 
in  the  field,  it  was  awarded  six  points  on  this  scale.  These  results  appear  in  the  third 
column  of  Table  1 . 


FACTORS  INFLUENCING  STABILITY 

Previous  reports  on  systems  development  have  noted  the  importance  of  various 
factors  that  influence  the  stability  of  system  acquisition  programs.  As  part  of  this 
research,  program  funding  uncertainty  and  cutbacks,  changes  to  the  system  require¬ 
ments  (e.g.,  changes  to  the  threat  the  system  was  being  designed  to  defeat),  and 
changes  in  key  military  personnel  representing  the  user  community  (e.g.,  Training 
and  Doctrine  Command)  were  all  examined  to  see  to  what  extent  any  or  all  of  these 
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TABLE  1.  SUMMARY  CASE  INFORMATION 


SYSTEM/CASE 

DEVELOPMENT 

DURATION 

(MONTHS) 

KEY 

OUTCOMES 

ACHIEVED 

(0-6) 

COMPLEXITY 

(LOW, 

MEDIUM, 

HIGH) 

APACHE  attack  helicopter 

108 

1 

High 

TADS/PNVS  (target 
acquisition  and 
designation/pilot’s  night 
vision  systems) 

~36 

3 

Medium 

MLRS  rocket  system 

33 

6 

Low 

ATACMS  missile  system 

37 

6 

Medium 

M40  chemical  protective 
mask 

~48 

2 

Low 

Mounted  microclimate 
cooler 

~24 

5 

Low 

M829-A1  armor-piercing 
kinetic  energy  tank 
ammunition 

~36 

6 

Low 

TOW-2A  (Tube-launched 
missile) 

48 

3 

Low 

AN/TAS  4  infrared  night 
sight 

~24 

4 

Low 

Joint  Stars  Ground  Station 

105 

1 

Medium 

Guardrail  common  sensor 

-24 

3 

Low 

PAC-2  (PATRIOT  anti¬ 
missile  system) 

~52 

2 

Medium 

HELLFIRE  missile  system 

~84 

3 

High 
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TABLE  2.  PROGRAM  STABILITY 


Type  of  Instability 

Question/Response 

Funding  uncertainty 

6  of  12  disagreed  strongly  that  there 
was  uncertainty  about  the  future  of 
project  funding. 

Project  slow-down 

8  of  1 3  projects  were  not  stopped  and 
restarted  or  slowed  down. 

Funding  cut-backs 

5  of  13  escaped  changes  or 
compromises  forced  by  cut-backs  in 
project  resources. 

Turn-over  in  Army  user  representatives 

All  projects  experienced  changes 
in  keyTRADOC  personnel  during 
development.  This  occurred  only  once 
or  twice  for  7 
of  13. 

Change  in  systems  requirements 

4  of  13  had  no  changes  in  systems 
requirements  during  development. 

Change  in  system  requirements 

4  projects  experienced  systems 
requirement  changes*  in  the  middle 
of  development;  in  3  these  changes 
occurred  late  in  development 

*  Responses  selected  as  many  periods  as  applicable  from  the  stages  of  planning; 
early,  mid-  and  late  development;  and  transition. 


“instabilities”  impacted  program  outcomes.  Table  2  contains  the  aggregate  results 
from  questions  that  were  used  in  the  research,  categorized  by  type  of  instability. 

A  somewhat  surprising  result  is  that  there  was  widespread  occurrence  of  all  these 
forms  of  instability  in  the  pre-DESERT  STORM  development  programs  studied 
here.  Looking  back  at  the  successful  performance  of  new  Army  systems  in  DESERT 
STORM,  one  might  think  that  their  development  benefited  from  strong  and  stable 
environments,  but  the  evidence  shows  that  the  external  environment  then  was  not 
unlike  what  one  might  find  today. 

FINANCIAL  UNCERTAINTY  AND  PROJECT  CUTBACKS 

Where  potential  or  actual  funding  changes  were  encountered  (“financial  un¬ 
certainty”),  it  appears  to  have  had  significant  consequences.  When  one  looks  at  the 
projects  that  are  reported  to  have  been  slowed  (i.e.,  experienced  a  lengthening  of  the 
planned  development  phase),  all  five  also  experienced  problems  due  to  financial  cut¬ 
backs.  By  comparison,  only  three  of  eight  that  were  not  slowed  experienced  problems 
due  to  cutbacks  (Table  3).  While  program  slow-down  may  be  caused  by  a  variety  of 
factors  besides  or  in  addition  to  budget  cuts,  once  slowed,  programs  seem  to  have 
continuing  financial  problems.  Given  that  cutbacks  are  often  the  first  signal  that  a 
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TABLE  3.  FUNDING  INSTABILITY  AND  FORCED  CHANGES  AND 
COMPROMISES 


A.  Often  uncertainty  about  future  of  project  funding 


Cut-backs  forced  changes 

Other 

responses 

Strongly 

disagree 

Changes  were  forced  by  cut-backs 

8 

0 

No  changes  forced  by  cut-backs 

2 

3 

Total 

10 

3 

Tau  B  =  0.683,  significant  at  .001. 


B.  Was  project  slowed  down? 

Cut-backs  forced  changes 

Slowed  down 

Kept  on 
schedule 

Changes  were  forced  by  cut-backs 

5 

3 

No  changes  forced  by  cut-backs 

0 

5 

Total 

5 

8 

Tau  B  =  0.688,  significant  at  .001. 


program’s  future  is  at  risk,  these  results  are  expected  and  provide  reassurance  that  the 
survey  respondents’ judgments  of  the  projects  are  consistent. 

The  earlier  LeanTEC  research  that  has  influenced  the  design  of  this  investigation 
found  that  there  were  some  ties  between  funding  stability  and  project  performance. 
Veteran  professionals  in  the  aerospace  industry  recalled  a  number  of  projects  that 
were  weakened  by  perceptions  that  project  funds  were  limited  or  at  risk.  They  sug¬ 
gested  that  when  funding  seemed  threatened,  development  team  engineers  had  a 


Financial  uncertainty  and  cutbacks  are  found  to  relate 
strongly  to  turnover. 


tendency  to  migrate  to  other,  more  stable  projects,  causing  turnover.  Being  able  to  bill 
to  multiple  engineering  charge  numbers  gives  the  individual  substantial  security  and 
control  over  his  or  her  work  if  the  primary  project  encounters  financial  cutbacks  or  is 
cancelled.  Other  interviewees  suggested  that  worry  about  continued  funding  led  man¬ 
agement  and  team  leaders  to  cut  back  on  staffing  or  otherwise  reduce  costs  to  stretch 
the  project  out.  Whatever  the  reasons,  respondents  were  confident  that  they  had  seen 
a  substantial  number  of  projects  where  funding  uncertainties  had  directly  contributed 
to  poor  team  performance  due  to  team  turnover  and  inadequate  staffing. 
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TABLE  4.  FUNDING  INSTABILITY  AND  STAFF  TURN-OVER 


A.  There  was  uncertainly  about  future  funding 

Members  turned  over? 

Other  responses 

Strongly  disagree 

Other  responses 

6 

0 

Disagree,  no  turnover 

1 

6 

Total 

7 

6 

Tau  B  =  0.667,  significant  at  .001.  Missing  data  for  one  case. 


B.  Cut-backs  forced  changes? 

Members  turned  over? 

Forced  changes 

No  changes  forced 

Other  responses 

6 

0 

Strongly  disagree,  no  turnover 

2 

5 

Total 

8 

5 

Tau  B  =  0.635,  significant  at  .001. 


The  results  from  this  present  study  of  Army  systems  support  that  view.  When 
one  looks  in  turn  at  how  these  questions  about  financial  stability  relate  to  other  key 
factors,  strong,  negative  relationships  are  found  with  staffing  stability  and  effective 
testing.  In  particular,  both  financial  uncertainty  and  cutbacks  are  found  to  relate 
strongly  to  turnover.  Of  six  projects  where  respondents  reported  little  or  no  funding 
uncertainty,  none  are  reported  to  have  had  turnover.  For  the  remaining  projects  where 
funding  was  more  uncertain,  six  of  seven  experienced  turnover  (top,  Table  4).  When 
one  compares  the  projects  that  didn’t  have  compromises  or  changes  forced  by  cut¬ 
backs  with  those  that  did,  the  results  show  that  all  five  projects  with  no  cutbacks  also 
had  no  turnover.  By  contrast,  six  of  the  eight  projects  that  experienced  cutbacks  also 
experienced  turnover  (bottom,  Table  4). 


A  NOTE  ON  STATISTICS 

Despite  the  limitation  that  the  analysis  only  includes  13  cases,  the  Tau  B 
statistical  test  appropriate  for  these  variables  (Blalock,  1960)  shows  that  the 
strength  of  the  relationship  between  the  two  variables  is  sufficiently  strong 
that  it  could  have  happened  by  chance  less  than  one  time  in  a  thousand  (p 
<  .001).  One  can  thus  have  confidence  that  the  reported  relationships  are 
statistically  meaningful  despite  the  small  number  of  programs  being  studied. 


Project  cutbacks  and  financial  stability  were  also  found  to  all  relate  to  the  ef¬ 
fectiveness  and  the  appropriate  timing  of  the  testing  used  in  the  program.  One  might 
readily  understand  that  slowing  projects  could  disrupt  testing  schedules,  and  that 
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TABLE  5.  FUNDING  INSTABILITY  AND  THE  TIMING  OF  TESTING 


A.  Was  project  slowed  down? 

Appropriate  timing  of  testing? 

Slowed  down 

Kept  on  schedule 

Other  responses 

4 

2 

Strongly  agree 

1 

6 

Total 

5 

8 

Tau  B  =  0.620,  significant  at  .004. 


B.  Cut-backs  forced  changes 

Appropriate  timing  of  testing? 

Occurred 

None 

Other  responses 

5 

1 

Strongly  agree 

3 

4 

Total 

8 

5 

Tau  B  =  0.610,  significant  at  .002. 


investment  in  testing  might  be  a  casualty  of  budget  cuts.  For  projects  slowed  during 
the  life  of  the  program,  four  of  five  cases  did  not  conduct  timely  testing  activities. 

For  projects  that  stayed  on  schedule,  six  of  eight  were  reported  as  having  conducted 
appropriate  testing  (top,  Table  5).  Cutbacks  could  cause  testing  changes  and  compro¬ 
mises,  which  could  lead  to  the  conclusion  that  testing  was  negatively  affected.  Four 
out  of  five  cases  that  did  not  have  any  changes  forced  by  cutbacks  also  had  appropri¬ 
ate  testing;  only  three  of  eight  programs  that  were  slowed  are  reported  confidently  as 
having  appropriately  timed  their  testing  activities  (bottom,  Table  5). 


Experiente  suggests  that  stretthing  projetts  disrupts 
sihedules,  and  that  tutbatks  and  thanges  often  lead  to  the 
need  to  repeat  old  test  protedures  or  design  new  ones . 


Experience  suggests  that  stretching  projects  disrupts  schedules,  and  that  cutbacks 
and  changes  often  lead  to  the  need  to  repeat  old  test  procedures  or  design  new  ones. 
That  and  the  presence  of  turnover  mean  that  revised  testing  programs  are  sometimes 
prepared  by  different  individuals  from  those  who  designed  the  system  and  supported 
its  integration.  Whatever  the  mechanisms,  the  general  conclusion  from  these  results 
is  that  funding  instability  seriously  affects  staffing  and  the  quality  of  testing,  which 
were  in  turn  shown  in  the  research  to  be  key  predictors  of  program  performance. 
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TABLE  6.  FINANCIAL  INSTABILITY  AND  PROJECT  PERFORMANCE 


Average  number  of  successful  outcomes  and  number 

Other 

responses 

Positive 

response* 

Signif.  at 

Stability  and  funding: 

Uncertainty  about 
project  funding? 

2.80  (10) 

5.67  (3) 

.001 

Project  ever  slowed  down? 

1.80  (5) 

4.50  (8) 

.001 

Turn-over  on 
development  team? 

2.00  (7) 

4.71  (6) 

.001 

*The  positive  responses  are,  in  order:  to  disagree  strongly  that  funding  was 
uncertain,  to  say  the  project  was  never  slowed  or  interrupted,  and  disagree  that 
there  was  any  team  turnover. 


The  overall  effect  of  financial-related  problems  on  project  performance  is  sum¬ 
marized  by  looking  at  the  average  number  of  successful  outcomes  (Table  6).  The 
six  cases  that  had  little  or  no  problem  with  uncertain  funding  had  an  average  of  5.67 
successes  out  of  a  possible  6.0;  those  that  did  face  this  uncertainty  averaged  2.80. 
Cases  that  were  never  slowed  averaged  4.50  successes  compared  with  1.80  for  those 
that  were  slowed.  The  impact  on  outcomes  of  the  relationship  between  the  funding 
environment  and  staffing  is  also  seen;  projects  that  experienced  turnover  averaged  2.0 
successes,  while  those  that  avoided  turnover  averaged  4.71.  It  might  be  argued  that 
financial  uncertainty  and  cutbacks  follow  when  projects  encounter  other  difficulties, 
in  which  case  one  could  expect  that  these  differences  in  averages  would  be  higher 
than  those  found  for  other  factors.  Nevertheless,  when  funding  problems  are  present, 
there  is  little  doubt  that  they  are  strongly  associated  with  turnover  and  poor  develop¬ 
ment  performance  for  these  DESERT  STORM  cases. 

SIGNIFICANT  CHANGES  IN  SYSTEMS  REQUIREMENTS 

As  with  other  forms  of  instability,  considerable  anecdotal  evidence  suggests  that 
significant  changes  in  systems  requirements  will  adversely  impact  program  outcomes, 
particularly  schedule  and/or  cost.  Consequently,  the  existence  of  this  evidence  makes 
experienced  project  managers  extremely  wary  of  permitting  any  changes  in  system 
requirements  to  occur.  Sometimes,  however,  actions  on  the  part  of  potential  adversar¬ 
ies,  referred  to  as  “changes  in  the  threat,”  can  force  the  issue. 

In  the  13  cases  studied,  only  3  reported  no  change  to  system  requirements  once 
a  system  concept  had  evolved,  and  only  4  reported  no  change  during  the  develop¬ 
ment  phase  of  the  project.  In  those  cases  that  experienced  change  during  develop¬ 
ment,  three  were  judged  to  have  required  “significant”  or  “major”  effort  to  make 
the  change,  while  the  remaining  six  only  required  “minor”  or  “very  minor”  effort. 
Moreover,  some  cases  experienced  multiple  instances  of  requirements  change  dur- 


122 


TABLE  7.  CHANGING  SYSTEMS  REQUIREMENTS  AND  PROJECT 
OUTCOMES 


A.  Frequency  of  changes  in  systems  requirements 

System  met  cost  goals? 

Several,  or 
Many  times 

None,  or  One 
or  two  times 

Fell  far  short  of  cost  goals 

1 

0 

Came  close  to  cost  goals 

3 

3 

Met  or  exceeded  cost  goals 

0 

6 

Total 

4 

9 

Tau  B  =  0.620,  significant  at  .003. 


B.  Frequency  of  changes  in  system  requirements 

Late  engineering  changes  after 
production  had  started 

Several,  or 
Many  times 

None,  or  One 
or  Two  times 

Significant  changes 

2 

0 

Minor  changes 

2 

6 

None,  almost  none 

0 

3 

Total 

4 

9 

Tau  B  =  0.537,  significant  at  .004. 


ing  development,  with  four  of  the  nine  describing  encountering  “several”  or  “many” 
changes.  The  remainder  reported  “no  change,”  or  only  “one  or  two  instances”  of 
change.  The  frequency  of  these  changes  seems  to  be  at  variance  with  the  relatively 
stable  doctrinal  and  operational  environment  prior  to  DESERT  STORM. 


When  thonging  requirements  ore  related  to  other  variables , 
the  results  support  the  tonventional  wisdom  that  suth 
thanges  are  tostly. 


When  changing  requirements  are  related  to  other  variables,  the  results  support 
the  conventional  wisdom  that  such  changes  are  costly.  Significant  correlations  were 
found  between  three  requirements  change  variables  and  several  of  the  outcome 
metrics.  None  of  the  four  projects  that  had  several  (three  cases)  or  many  (one  case) 
requirements  changes  met  their  cost  goals  (top,  Table  7),  and  none  of  the  four  avoided 
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TABLE  8.  STABILITY  OF  SYSTEMS  REQUIREMENTS  AND  PROJECT 
PERFORMANCE 


Average  number  of  successful  outcomes 

Mid-development 
requirements  change? 

Changes 

occurred 

No  changes 

Signif.  at 

2.00  (4) 

4.11  (9) 

.005 

Frequency  of  systems 
requirements  change? 

Several  or 
Many  times 

None,  or  One 
or  two  times 

Signif.  at 

1.50  (4) 

4.33  (9) 

.006 

late  engineering  changes  (bottom,  Table  7).  For  those  that  had  none  or  only  one  or 
two  systems  requirements  changes,  six  of  nine  met  their  cost  goals,  and  three  of  nine 
avoided  (even  minor)  late  engineering  changes.  Weaker  but  similar  negative  differ¬ 
ences  on  other  outcomes  (not  shown)  are  found  among  those  projects  that  experi¬ 
enced  more  requirements  changes. 

One  can  see  the  overall  effects  of  the  frequency  and  timing  of  systems  require¬ 
ments  changes  by  looking  at  how  many  successful  outcomes  these  projects  had  on 
average.  The  respondents  were  asked  in  what  stage  of  development  the  requirements 
had  changed,  and  it  was  found  that  the  negative  impact  on  the  average  rates  of  suc¬ 
cess  was  greatest  when  changes  had  occurred  in  mid-development  (typically  after 
the  Critical  Design  Review).  As  shown  in  Table  8,  the  four  projects  that  had  systems 
requirement  changes  in  mid-development  had  only  an  average  of  2.0  positive  perfor¬ 
mance  outcomes,  compared  to  4. 1 1  average  positive  outcomes  of  those  that  did  not 
change  in  mid-development. 

The  impact  is  even  greater  when  one  looks  at  the  frequency  of  requirements 
changes  and  the  average  number  of  successful  outcomes.  The  four  projects  said  to 
have  seen  systems  requirements  changes  several  or  many  times  during  development 
averaged  only  1.50  successful  outcomes,  compared  to  4.33  successes  among  those 
projects  that  had  no  or  only  one  or  two  systems  requirements  changes. 

TURNOVER  IN  KEY  USER  PERSONNEL 

The  U.S.  Army  Training  and  Doctrine  Command  (TRADOC)  is  responsible  for 
determining  the  requirements  that  Army  materiel  must  meet  in  order  to  have  utility 
on  the  battlefield.  A  senior  TRADOC  staff  member  (typically  a  colonel)  serves  as 
the  alter  ego  of  the  Project  Manager  in  interpreting  these  requirements  as  they  are 
translated  into  system  technical  requirements  during  the  acquisition  process.  This  key 
individual  may  also  play  a  critical  role  in  preserving  the  planned  funding  for  the  sys¬ 
tem  development  by  persuading  more  senior  TRADOC  leaders  to  strongly  reaffirm 
the  need  for  the  system  when  budget  cuts  are  threatened  or  problems  are  encountered 
in  the  system  development  that  increase  cost  or  stretch  schedule.  The  frequency  and 
timing  of  turnover  in  key  TRADOC  personnel  were  examined  to  determine  influence 
on  project  outcomes. 
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TABLE  9.  CONTINUITY  OF  TRADOC  STAFFING 


Did  TRADOC  change  during  early  development? 

Operational  problems  in  the  field? 

Yes 

No 

Field  problems  limited  effectiveness 

4 

1 

Deployed  at  no  loss  of  effectiveness 

1 

4 

Exceeded  expectations 

0 

7 

Total 

5 

7 

Tau  B  =  0.630,  significant  at  .001.  Data  are  available  on  12  cases  for  this 
TRADOC  question. 


All  of  the  respondents  reported  that  their  projects  had  experienced  some  turnover 
in  key  TRADOC  personnel.  Only  two  reported  no  key  TRADOC  personnel  changes 
during  the  development  phase  of  the  project.  Such  regular  change  is  consistent 
with  the  military  reassignment  cycle.  The  timing  of  when  TRADOC  key  personnel 
turnover  occurred  correlated  with  several  of  the  outcome  metrics,  most  notably  with 
the  extent  to  which  the  system  met  expectations  when  used  on  the  battlefield  during 
DESERT  STORM.  Table  9  shows  the  negative  impact  of  staff  change  early  in  devel¬ 
opment  on  system  performance  on  the  battlefield.  Key  TRADOC  personnel  changed 
for  five  cases  during  early  development,  and  four  of  those  projects  subsequently 
encountered  operational  field  problems.  Where  there  was  no  early  TRADOC  change, 
only  one  of  seven  projects  was  not  as  effective  as  expected. 


Cases  that  experienced  no  TRADOC  ihanges  in  early 
development ;  however,  are  seen  to  be  substantially  more 
successful  at  an  average  of  4.29  successful  outcomes. 


The  only  two  cases — Night  Sight  and  the  M829A1  sabot  round — that  did  not 
experience  TRADOC  changes  during  the  development  phase  are  the  same  two  cases 
that  the  respondents  felt  exceeded  performance  expectations  in  the  field. 

To  further  examine  the  possible  impact  of  TRADOC  changes  at  different  stages 
of  the  project,  one  can  again  look  at  the  average  number  of  goals  met  for  the  differ¬ 
ent  projects.  No  relationships  of  consequence  are  found  between  TRADOC  change 
in  mid-  and  late  development  and  the  average  number  of  successful  outcomes.  Cases 
that  experienced  no  TRADOC  changes  in  early  development,  however,  are  seen  to  be 
substantially  more  successful  at  an  average  of  4.29  successful  outcomes,  compared  to 
an  average  of  2.40  for  those  that  did  have  TRADOC  changes  at  that  time. 


125 


TABLE  10.  TRADOC  CHANGES  AND  PROJECT  PERFORMANCE 


Average  number  of  successful  outcomes 

TRADOC  change  during 
early  development? 

Staff 

change 

No  change 

Signif.  at 

2.40  (5) 

4.29  (7) 

.011 

TABLE  11.  CHANGING  SYSTEMS  REQUIREMENTS  AND  KEY  TRADOC 
PERSONNEL 


TRADOC  change  in  early  stages*  of  project? 

Did  system  requirements 
change  during  mid 
development? 

No  change 

In  one 
stage 

In  both  stages 

No 

5 

3 

0 

Yes 

0 

2 

2 

Total 

5 

5 

2 

Tau  B  =  0.0685,  significant  at  0.001  *pre-deveiopment  or  early  development 


One  would  expect  that  the  doctrinal  and  operational  underpinning  for  the  systems’ 
requirements  should  have  been  relatively  constant  (during  this  late  Cold  War  period), 
but  the  authors  found  a  substantial  number  of  changes  in  systems  requirements  for 
these  cases.  The  question,  therefore,  lingers:  Did  the  changes  in  TRADOC  personnel 
found  here  somehow  play  a  role?  New  personnel  may  not  feel  that  their  predecessors 
had  correctly  defined  the  threat’s  implications  on  requirements,  or  having  not  been  a 
party  to  earlier  discussions,  been  more  willing  to  pursue  changes  suggested  by  new 


The  most  damaging  requirements  thanges  are  those  that 
oaur  in  mid-development 


knowledge  and  events.  This  suggestion  raises  the  possibility  that  TRADOC  person¬ 
nel  change  could  have  adverse,  indirect  effects  by  somehow  permitting  changes  in 
systems  requirements  that  have,  in  turn,  a  negative  impact  on  project  performance. 

This  study  analyzed  the  relationship  between  early  TRADOC  staff  changes  and 
shifts  in  systems  requirements  and  found  some  support  for  that  view.  As  noted  previ¬ 
ously,  the  results  in  Table  10  suggest  that  the  most  damaging  requirements  changes 
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TABLE  12.  PROJECT  DURATION,  INSTABILITY  AND  FIELD  PERFORMANCE 


How  often  did  systems  requirements  change? 

Three  years 
or  less 

Four  years 
or  more 

Never 

4 

0 

Once  or  twice 

3 

2 

Several  or  many  times 

0 

4 

Total 

7 

6 

Tau  B  =  .723,  significant  at  .001. 


Did  system  have  operational  problems  in  the  field? 

Did  not  meet  expectations 

1 

4 

Met  expectations 

4 

2 

Exceeded  expectations 

2 

0 

Total 

7 

6 

Tau  B  =  -.556,  significant  at  .003. 


are  those  that  occur  in  mid-development.  To  look  for  a  relevant  relationship,  one 
can  aggregate  TRADOC  changes  by  asking  whether  there  were  key  staff  changes 
during  the  technology  selection/planning  period  or  during  early  development  (the 
two  earliest  time  slices  in  the  project).  One  can  then  compare  the  projects  that  did  or 
did  not  have  these  early  key  TRADOC  personnel  changes.  Table  1 1  shows  that  five 
projects  avoided  early  TRADOC  changes,  and  none  of  the  five  had  mid-development 
changes  in  systems  requirements.  Of  the  five  projects  that  had  TRADOC  changes 
in  one  of  these  early  stages,  two  experienced  requirements  changes.  The  two  proj¬ 
ects  that  had  TRADOC  changes  in  both  planning  and  early  development  saw  later 
requirements  changes  in  mid-development.  It  would  appear  that  TRADOC  turnover 
in  the  early  stages  of  projects  is  in  some  way  related  to  mid-development  changes  in 
systems  requirements. 


CONCLUSIONS 

Taken  together,  these  several  relationships  strongly  suggest  that  stability  of 
program  resources,  staffing,  and  objectives  is  a  very  powerful  influence  on  the  rela¬ 
tive  success  of  projects.  Certainly,  as  has  been  noted,  a  wealth  of  anecdotal  evidence 
suggests  that  this  should  be  the  case.  In  reflecting  on  this  array  of  instabilities  that 
could  impact  a  system  development,  it  became  clear  that  they  had  at  least  one  thing 
in  common.  That  is,  the  longer  a  system  stayed  in  development,  the  greater  chance  it 
had  to  experience  one  or  more  of  these  program  destabilizing  events. 

Support  for  this  view  is  found  when  the  occurrence  of  cutbacks  and  systems  re¬ 
quirements  changes  are  related  to  the  duration  of  the  13  development  projects  studied 
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TABLE  13.  LENGTH  OF  PROJECT  DEVELOPMENT  AND  PROJECT 
PERFORMANCE 


Average  number  of  successful  outcomes 

Project  duration? 

Three  years 
or  less 

Four  years 
or  more 

Signif.  at 

4.71 

2.00 

.001 

here.  The  projects  divided  cleanly  between  7  that  took  37  months  or  less,  and  6  that 
took  48  months  or  more.  When  the  frequency  of  systems  requirements  changes  is 
considered,  longer  duration  clearly  allows  more  time  for  changing  external  conditions 
and  priorities  to  lead  to  changes  in  systems  requirements.  Four  of  seven  of  the  shorter 
projects  never  experienced  any  changes  in  requirements,  and  the  remaining  three  only 
experienced  minimal  changes.  None  of  the  six  projects  that  ran  four  years  or  longer 
avoided  requirements  changes,  two  saw  changes  once  or  twice,  and  four  experienced 
either  several  or  many  changes.  The  pattern  is  quite  strong  and  statistically  significant 
despite  the  limited  number  of  cases  (top,  Table  12). 

Given  that  longer  schedules  increase  the  risk  of  encountering  instabilities,  it 
is  then  not  surprising  that  project  duration  is  also  negatively  related  to  achieving 
desirable  project  outcomes.  For  example  (bottom,  Table  12),  one  of  the  strongest 
relationships  between  development  duration  and  project  outcomes  is  found  for  how 
the  system  performed  in  the  field.  Six  of  the  seven  systems  developed  in  (about)  three 
years  or  less  met  or  exceeded  expectations  when  they  were  deployed  in  DESERT 
STORM,  compared  to  only  two  of  six  of  the  longer  projects.  The  effect  on  all  six 
outcome  questions  is  seen  in  Table  13  with  the  projects  requiring  longer  develop¬ 
ment  time  averaging  success  on  only  two  outcomes,  where  the  shorter  development 
projects  had  an  average  4.7  successful  outcomes. 

The  sensitivity  of  this  central  conclusion  to  project  complexity  was  examined 
using  a  measure  of  relative  complexity  (Table  1)  developed  for  this  purpose.  More 
complex  projects  often  require  longer  development  cycles  and  are  more  likely  to 
experience  funding  difficulties.  Flowever,  complexity  is  much  more  weakly  related 
with  staffing  turnover  than  is  project  duration,  and  differences  in  complexity  are  not 
found  to  be  at  all  related  with  changes  of  TRADOC  personnel  or  changes  in  systems 
requirements.  Complexity  is  also  not  related  to  testing  quality  and  timeliness;  both 
of  these  factors  are  strongly  (and  positively)  correlated  with  outcomes.  Importantly, 
duration  alone  is  more  strongly  related  than  complexity  to  the  number  of  successful 
outcomes.  Whether  or  not  complexity  plays  a  role,  project  duration  has  a  strong  effect 
on  outcomes  independent  of  the  influence  of  complexity. 

The  evidence  here  is  that  time  is  not  an  ally  of  systems  development.  The  pas¬ 
sage  of  time  and  the  inevitable  intrusion  of  new  knowledge  open  the  door  to  new 
pressures.  Financial  uncertainty  is  created  in  part  by  the  need  for  resources  for  newer 
projects,  often  leading  to  staffing  doubts  about  the  current  program  and/or  the  ap¬ 
pearance  of  new  opportunities  where  key  personnel  are  also  needed.  Or,  simple 
career  progression  incentives  lead  people  after  time  to  move  on,  taking  knowledge 
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and  experience  with  them.  An  obvious  conclusion  is  that  programs  undertaken  for 
around  three  years  or  less  involve  less  risk  of  destabilizing  factors  outside  the  control 
of  program  managers.  These  data  then  suggest  that  it  is  better  to  organize  and  bud¬ 
get  projects  for  shorter  time  frames  whenever  possible.  If  projects  must  be  planned 
to  take  four  years  or  longer  in  development,  this  research  suggests  that  one  should 
recognize  that  longer  projects  open  the  way  for  substantial  pressures  that  may  be 
difficult  to  resist,  and  then  plan  to  deal  with  instability.  Of  the  several  factors  consid¬ 
ered  in  this  study,  the  areas  that  should  receive  particular  attention  in  longer  projects 
are  the  need  not  to  take  advantage  of  the  expanded  opportunity  to  change  systems 
requirements,  to  keep  key  user  representatives  in  their  positions  during  the  critical 
early  program  phases,  to  provide  incentives  and  career  planning  to  avoid  turnover 
in  key  development  personnel,  and  to  ensure  that  cutbacks  and  rescheduling  do  not 
compromise  testing  regimes. 
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HOW  TO  MAKE  INCENTIVE 
AND  AWARD  FEES  WORK 

Alan  S.  Gilbreth  and  Sylvester  Hubbard 


Incentive  and  award  fees  are  not  driving  performance  outcomes  as  originally 
envisioned.  In  2005,  the  Government  Accountability  Office  (GAO)  identified 
an  apparent  disconnect  between  the  use  of  certain  measures  like  incentive 
and  award  fees  and  expected  outcomes  in  major  acquisitions  (GAO,  2005). 

In  response  to  the  GAO  report,  the  authors  conducted  a  research  effort  to 
better  understand  where  incentive  and  award  fees  had  favorable  impact  on 
performance  outcomes  and  why.  This  article  summarizes  the  findings  of  the 
research,  highlights  several  organizations  that  clearly  used  techniques  that 
drove  favorable  outcomes,  and  provides  recommendations  and  take-aways 
that  will  promote  effective  and  efficient  incentive  and  award  fee  programs. 

Recently,  major  acquisition  activities  have  received  significant  attention  due  to 
failure  to  achieve  desired  outcomes,  such  as  meeting  cost  and  schedule  goals 
and  delivering  desired  capabilities.  The  GAO  has  looked  closely  at  the  effec¬ 
tiveness  of  incentives  in  DoD  contracts.  They  selected  a  sample  of  93  contracts  from 
a  study  population  of  597  DoD  incentive  and  award  fee  contracts  that  were  active 
between  fiscal  years  1999  and  2003.  Fifty-two  contracts  (56  percent  of  the  sample) 
were  award  fee,  27  contracts  (29  percent  of  the  sample)  were  incentive  fee,  and  14 
(15  percent  of  the  sample)  included  both  incentive  and  award  fee  provisions.  The 
GAO  published  their  report  in  December  2005  (GAO-06-66)  and  asserted  that  “DoD 
has  paid  billions  in  incentive  and  award  fees  without  favorably  influencing  perfor¬ 
mance  outcomes”  (GAO,  2005). 

In  response  to  the  GAO  report,  then  Under  Secretary  of  Defense  for  Acquisition, 
Technology  and  Logistics  Ken  Krieg  directed  the  Defense  Acquisition  University 
(DAU)  to  conduct  a  research  effort  to  better  understand  where  incentive  and  award 
fees  had  a  favorable  impact  on  performance  outcomes  and  what  made  these  programs 
effective.  The  goal  of  the  research  was  not  to  validate  or  reclama  the  GAO  study. 
Instead,  the  research  effort  was  to  investigate  where  the  acquisition  community 
implemented  award  and  incentive  fee  practices  that  had  a  favorable  impact  on  perfor¬ 
mance  outcomes  and  could  be  adopted  as  best  practices  throughout  the  acquisition 
community.  The  authors  collaborated  with  faculty  members  from  other  DAU  regions 
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on  this  research  (Tremaine,  2007)  and  specifically  conducted  surveys  of  program  of¬ 
fices  located  at  Tank- Automotive  and  Armaments  Command  Life  Cycle  Management 
Command  (TACOM  LCMC)  in  Warren,  MI;  Air  Mobility  Command  (AMC)  at  Scott 
Air  Force  Base  (AFB),  IL;  and  Aeronautical  Systems  Center  (ASC)  at  Wright-Patter- 
son  AFB,  OH.  The  program  offices  interviewed  (Table  1)  were  recommended  by  their 
major  commands  as  having  an  effective  and  efficient  award  or  incentive  fee  program. 


TABLE  1.  PROGRAM  OFFICES  INTERVIEWED 


ORGANIZATION 

PROGRAM  OFFICE 

TACOM  Life  Cycle  Management 
Command  (LCMC) 

Army  Future  Combat  Systems  (FCS) 

TACOM  LCMC 

Army  Total  Integrated  Engine 
Revitalization  (TIGER) 

TACOM  LCMC 

Army  Biological  Detection  System 

Aeronautical  Systems  Center  (ASC) 

B-2  Aircraft-Radar  Modernization 
Program-Frequency  Change 

ASC 

C-17  Aircraft-Sustainment 

ASC 

F-15  Aircraft-Suite  6  Software  Upgrade 
for  A,  D,  &  E  Models 

ASC 

F-16  Aircraft-Operational  Flight  Program 
Development 

ASC 

Global  Hawk-Unmanned  Aerial  Vehicle 

Air  Mobility  Command  (AMC) 

Global  Transportation  Network  (GTN) 

AMC 

Air  Mobility  Command  Contractor 

Tactical  Terminal  Operations 

The  research  team  conducted  each  survey  interview  with  the  program  or  project 
manager  and  contracting  officer  for  the  particular'  program  or  project.  Even  though 
the  team  did  not  meet  individually  with  the  industry  representatives,  contractor  per¬ 
spectives  were  considered  an  important  element  of  this  research. 

During  midsummer  2006,  DAU  hosted  an  Industry  Day  at  Fort  Belvoir,  VA. 
Eighteen  senior-level  industry  representatives  from  throughout  the  United  States 
came  to  Fort  Belvoir  and  spoke  candidly  about  their  experience  with  incentive  con¬ 
tracts.  Table  2  represents  a  summary  of  their  consensus  views. 

This  article  highlights  the  award  and  incentive  fee  findings  that  resulted  in  favor¬ 
able  outcomes  and  examines  the  innovative  use  of  award  and  incentive  fee,  as  demon¬ 
strated  in  several  acquisition  programs. 
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TABLE  2.  SUMMARY  OF  CONSENSUS  VIEWS 


1. 

Government  construction  of  the  award  fee  plan  (including  metrics, 
incentives,  etc.)  may  not  link  with  the  offeror’s  proposed  solution 
or  motivations 

2. 

Industry  welcomes  the  use  of  base  fee  to  better  delineate  the  difference 
between  “best  efforts”  (e.g.  base  fee)  and  “excellence”  (e.g.  award  fee) 

3. 

In  some  cases,  the  government  does  not  follow  its  own  policies  on 
award  fee 

4. 

On  occasion,  award  fee  evaluation  criteria  are  poorly  explained  or 
justified  and  communication  of  award  fee  goals  and  criteria  are  not 
clearly  explained 

5. 

It  is  difficult  to  establish  the  relationship  between  awards  for  month-to- 
month  activities  to  the  goals  of  a  multiple-year  program.  The  linkage  is  not 
always  apparent 

6. 

Administration  of  award  fee  criteria  can  change  in  post  award  and  create 
problems  during  contract  execution 

7. 

Government  personnel  are  not  always  adequately  trained  in  managing 
award  fee  contracts 

8. 

Post-award  administration  of  award  fee  contracts  is  time  and 
resource  intensive 

9. 

Desired  outcomes  are  not  always  driven  by  the  award  fee  because  of 
insufficient  funds  available  and  subjectivity  of  the  final  evaluation 

10. 

There  are  sometimes  inconsistencies  in  the  timing  of  the  award  fee  in  line 
with  the  evaluation  criteria 

11. 

Government  and  contractors  sometimes  have  different  perceptions  of  the 
purpose  of  award  fees 

12. 

In  some  cases,  there  is  government  failure  to  understand  the  economics 
of  defense  contracting  and  its  impact  on  government  contractors 

13. 

From  time  to  time,  there  is  inappropriate  use  of  award  fee  contracts 

14. 

Requirements  are  sometimes  too  subjective  and  do  not  measure 
outcomes  that  are  sought  by  DoD 

135 


FINDINGS 


The  eight  findings  presented  here  are  the  result  of  survey  data  from  several  program 
offices  and  other  support  material,  which  allowed  the  authors  to  draw  conclusions  that 
the  information  and  techniques  would  help  acquisition  organizations  drive  favorable 
outcomes  in  their  incentive  programs.  The  intention  is  to  have  acquisition  organizations 
review  all  of  the  findings  in  a  holistic  manner  to  truly  drive  favorable  outcomes. 

REGULATORY  GUIDANCE  RELATING  TO  INCENTIVE  CONTRACTING  IS 
SUFFICIENT 

Program  managers  and  contracting  officers  interviewed  supported  the  view  that 
the  Federal  Acquisition  Regulation  (FAR)  and  the  Defense  Federal  Acquisition  Regu¬ 
lation  Supplement  (DFARS)  were  clear  and  complete.  The  problem  is  that  in  many 
cases,  these  regulations  are  not  followed  when  a  specific  situation  occurs  that  requires 
a  decision  regarding  incentive  contracting.  For  example,  cost-plus-award-fee  (CPAF) 
contracts  certainly  have  their  place  in  an  acquisition  professional’s  tool  kit,  but  FAR 
16.405-2(b)  states  that  a  “cost-plus-award-fee  contract  is  suitable  for  use  when  the 
work  to  be  performed  is  such  that  it  is  neither  feasible  nor  effective  to  devise  pre¬ 
determined  objective  incentive  targets  applicable  to  cost,  technical  performance,  or 


Fundamental  to  making  a  good  business  detision  on  whith 
incentive  tool  will  yield  the  desired  behavior  is  a  thorough 
knowledge  of  program  risk  and  contract  types,  and 
particularly  how  contract  type  impacts  risk. 


schedule”  (FAR,  April  2008).  It  also  goes  on  to  further  define  more  specific  consider¬ 
ations  for  application.  The  logic  behind  the  FAR  verbiage  is  that  the  first  choice  to  be 
considered  is  not  award  fee,  but  incentive  fee  using  predetermined  objective  incentive 
targets  applicable  to  cost,  technical  performance,  or  schedule.  Given  this  logic,  one 
could  reasonably  conclude  that  the  DoD  would  use  more  incentives  with  predeter¬ 
mined  objectives  rather  than  award  fee  in  their  contracts. 

The  same  logic  from  the  paragraph  above  applies  to  fixed-price  incentive  con¬ 
tracts  versus  fixed-price  contracts  with  award  fee.  FAR  16.404(a)  states,  “award-fee 
provisions  may  be  used  in  fixed-price  contracts  when  the  government  wishes  to 
motivate  a  contractor  and  other  incentives  cannot  be  used  because  contractor  perfor¬ 
mance  cannot  be  measured  objectively”  (FAR,  April  2008).  Again,  given  this  logic, 
DoD  would  use  more  incentive  contracts  with  predetermined  objectives  rather  than 
fixed  price  contracts  with  award  fees. 
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Fundamental  to  making  a  good  business  decision  on  which  incentive  tool  will 
yield  the  desired  behavior  is  a  thorough  knowledge  of  program  risk  and  contract 
types,  and  particularly  how  contract  type  impacts  risk.  Part  16  of  the  FAR  has  an 
excellent  treatment  of  contract  types.  The  use  of  appropriate  contract  types  and 
associated  incentives  represents  a  foundational  communication  and  risk  tool  for  the 
acquisition  professional. 

This  first  finding  might  lead  one  to  conclude  that  if  award  fee  requires  additional 
administrative  effort,  people,  and  cost,  and  should  only  be  used  when  the  work  to  be 
performed  is  such  that  it  is  neither  feasible  nor  effective  to  devise  objective  incen¬ 
tive  targets,  then  why  use  award  fee  at  all?  The  answer  is  there  are  times  when  only 
a  subjective  incentive  will  achieve  the  desired  goal  or  outcome.  Subjective  measures 
may  be  appropriate  to  drive  critical  processes,  management  responsiveness,  and  other 
unquantifiable  behaviors.  In  those  situations,  award  fee  may  be  used  as  long  as  the 
expected  benefits  are  sufficient  to  warrant  the  additional  administrative  effort  and 
cost.  Due  to  the  complexity  and  difficulties  associated  with  award  fee  contracts,  most 
of  the  findings  in  the  remainder  of  the  article  will  focus  on  award  fee. 

CONSISTENT  INCENTIVE  PHILOSOPHY 

Every  government  buying  activity  should  have  a  clearly  defined  incentive  philos¬ 
ophy  and  broadly  communicate  that  philosophy  to  its  personnel  and  potential  sources. 
Remember,  however,  that  consistency  and  flexibility  are  not  mutually  exclusive.  A 
consistent  incentive  policy  can  and  should  provide  flexibility  to  arrive  at  an  appropri¬ 
ate  incentive  strategy  that  suits  the  circumstances  of  each  contract  action.  This  may 
cause  one  to  use  multiple  incentive  types,  as  is  occurring  in  many  successful  incen¬ 
tive  programs. 

Use  the  contract  and  incentive  plan  to  communicate  the  philosophy  and  overall 
mission  objectives  to  the  contractor.  The  result  will  be  consistent  communication 
between  the  buying  activity  and  the  contractor  community. 

MULTIPLE  INCENTIVE -TYPE  CONTRACTS— BUILD  TO  NEED 

According  to  the  majority  of  organizations  interviewed,  the  combination  of 
objective  and  subjective  measures  indicated  the  strongest  correlation  to  expected 
outcomes.  The  subjective  measures  were  only  used  when  objective  measures  alone 
would  not  achieve  the  desired  outcome  to  the  same  level. 

BASE  FEE— AN  INTEGRAL  PART  OF  COST-PLUS-AWARD-FEE  CONTRACTS 

The  research  team  found  that  organizations  that  were  successfully  implementing 
CPAF  understood  the  puipose  and  value  of  using  a  base  fee  as  a  leverage  tool  for  the 
proper  use  of  the  award  fee  portion  of  CPAF.  The  DFARS  216.405-2(c)  (iii)  currently 
states,  “the  base  fee  shall  not  exceed  3  percent  of  the  estimated  cost  of  the  contract 
exclusive  of  fee”  (DFARS,  July  2006). 

Using  a  base  fee  allows  the  government  to  reward  the  contractor’s  “best  efforts” 
without  compromising  the  intent  of  the  award  fee  portion.  In  a  scenario  where  the 
contract  calls  for  no  base  fee,  contractors  could  provide  their  best  efforts  for  the 
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award  fee  period  and  receive  no  fee  if  they  did  not  achieve  “excellent”  performance. 
Best  efforts  refers  to  the  contractor’s  responsibility  to  meet  the  contract  requirements 
under  the  terms  of  the  contract.  When  no  base  fee  is  provided  in  the  contract,  it  puts 
pressure  on  the  government  to  provide  some  portion  of  the  award  fee  for  best  efforts. 
This  opens  the  door  to  more  administratively  burdensome  measures  like  the  2003  rule 
in  DFARS  216.405-2(b)(3)  that  permits  provisional  award  fee  payments.  Properly 
using  base  fee  could  alleviate  the  need  for  provisional  award  fee  payments.  When  a 
portion  of  the  award  fee  is  paid  for  best  efforts,  it  opens  the  government  to  criticism 
by  oversight  agencies  for  providing  award  fee  for  less-than-excellent  performance. 


When  no  base  fee  is  provided  in  the  contract,  it  puts 
pressure  on  the  government  to  provide  some  portion  of  the 
award  fee  for  best  efforts. 


If  a  base  fee  is  in  the  CPAF  contract,  the  contractor  will  receive  some  amount  above 
costs  for  best  efforts.  For  example,  the  F-15  Systems  Program  Office  (SPO)  commu¬ 
nicated  with  the  contractor  that  they  were  only  going  to  pay  award  fee  for  excellence, 
and  in  their  deliberations  the  contractor  asked  for  a  3  percent  base  fee.  The  F-15  SPO 
agreed  with  the  contractor,  the  logic  being  that  it  would  provide  the  SPO  the  leverage 
they  needed  to  only  pay  the  award  fee  portion  for  excellent  performance.  To  further 
explain  the  leverage  aspect  of  the  base  fee  in  a  CPAF  contract,  the  base  fee  should  be 
paid  regardless  of  performance  on  the  contract  as  long  as  the  contract  is  not  terminated. 
On  government  cost  reimbursement-type  contracts,  the  actual  payment  of  a  base  fee 
typically  accompanies  a  contractor’s  regular  invoice  for  reimbursement  of  costs. 

Other  programs  that  have  used  CPAF  successfully  appear  to  be  recognizing  the 
value  of  using  a  base  fee.  To  distinguish  excellent  performance  from  best  efforts,  the 
Global  Flawk  SPO  revised  their  contract  to  include  a  3  percent  base  fee. 

The  defense  contractors  interviewed  at  Fort  Belvoir  agreed  in  principle  that  in 
CPAF  contracts,  some  amount  of  base  fee  for  their  best  efforts  would  more  clearly 
delineate  the  difference  between  payment  for  best  efforts  and  payment  for  excellent 
performance  (award  fee),  thereby  reducing  the  criticism  concerning  the  use  of  award 
fee  contracts. 

EVALUATION  PERIODS— LINKAGE  IS  KEY 

One  common  data  point  for  determining  evaluation  periods  is  linkage  to  signifi¬ 
cant  events.  Using  events  or  outcomes  to  determine  award  fee  periods  instead  of  time 
ensures  that  the  award  fee  period  will  be  appropriate  to  the  events  or  outcomes.  When 
award  fee  is  based  on  time,  there  is  an  assumption  of  what  events  or  outcomes  will 
occur  in  that  time  period;  and  that  assumption  also  drives  the  amount  of  award  fee  for 
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the  period.  In  complex  contracts,  schedules  often  slip,  so  if  the  award  fee  period  is 
based  on  time,  and  a  significant  event  or  outcome  slips  to  another  award  fee  period, 
the  government  could  have  an  inappropriate  amount  of  award  fee  available  for  the 
wrong  reasons,  thus  not  achieving  the  objectives  of  the  award  fee  plan. 

Examples  of  evaluation  periods  linked  to  significant  events  are:  (a)  The  F-15 
program  tied  the  award  fee  periods  to  program  milestones  so  the  award  fee  periods 
varied  from  6  to  9  months  depending  on  when  the  milestones  were  reached;  (b) 
Global  Hawk  added  alignment  with  the  contractor’s  fiscal  year;  and  (c)  Both  the 
Army’s  Future  Combat  Systems  and  Total  Integrated  Engine  Revitalization  Program 
used  cost-plus-fixed-fee  (CPFF)  contracts  with  incentive  events. 

The  events  were  tied  to  contract  outcomes  and  varied  with  each  period.  These 
examples  clearly  indicate  that  when  possible,  evaluation  periods  should  be  linked  to 
events  or  outcomes  rather  than  time. 

FEEDBACK— SHOULD  BE  FREQUENT  AND  UNAMBIGUOUS 

Feedback  in  incentive  contracting  is  important,  particularly  in  contracts  with  an 
award  fee  element  because  they  require  so  much  administration.  The  administrative 
burden  can  seem  to  be  bureaucratic  and  excessive.  However,  the  research  results 
showed  that  monthly  feedback  within  the  joint  government  and  contractor  team  was 
more  effective  than  quarterly  or  biannual  feedback.  Continuous  and  open  dialogue  at 
both  junior  and  senior  levels  led  to  early  discovery  and  timely  reconciliation  of  many 
known  issues  and  helped  keep  the  program  on  track.  Many  program  offices  empha- 


Continuous  and  open  dialogue.Jed  to  early  distovery  and 
timely  retondliation  of  many  known  issues  and  helped  keep 
the  program  on  trark. 


sized  that  they  were  very  open  with  the  contractor;  therefore,  the  contractor  was  never 
surprised.  Additionally,  some  programs  instituted  certain  techniques  like  “emphasis 
letters”  during  the  award  fee  periods  to  stress  the  importance  of  certain  outcomes 
or  events.  One  particular  program  office  employed  what  they  called  a  “barometer 
report”  during  interim  reviews  to  ensure  that  information  from  monitors  was  readily 
available  to  manage  at  critical  junctures. 

Several  program  offices  found  it  necessary  to  establish  a  glossary,  particularly 
for  terms  related  to  critical  outcomes  or  events  in  incentives.  The  glossary  was  used 
as  a  tool  to  improve  communication  during  the  evaluation  briefings.  Glossaries  were 
also  beneficial  when  team  member  changes  occurred.  A  good  example  for  the  use  of 
glossaries  occurred  in  the  B-2  SPO  on  the  Radar  Modernization  Program-Frequency 
Change  contract.  The  program  was  in  the  systems  development  and  demonstration 
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(SDD)  phase,  and  there  was  a  requirement  in  the  contract  that  the  radar  make  first 
flight  to  demonstrate  success.  The  government  understood  first  flight  to  mean  aboard 
an  aircraft  and  demonstrating  functionality.  The  contractor  understood  a  successful 
first  flight  to  be  only  aboard  an  aircraft.  They  never  intended  the  radar  to  demonstrate 
functionality  on  first  flight.  Contracts  that  include  incentives  must  be  unambiguous, 
and  glossaries  are  a  good  tool  to  prevent  misunderstandings. 

CONDUCT  TRAINING  AND  USE  EXPERIENCED  PERSONNEL 

Nothing  seems  to  have  a  more  dramatic  impact  in  DoD  than  training  and  experi¬ 
ence.  One  learns  from  successes  and  failures  in  the  field,  and  then  makes  adjustments. 
Incentive-type  contracts  are  no  different.  Many  of  the  program  offices  responding  to 
the  survey  incorporated  the  practical  incentive/award  fee  experience  of  personnel  al¬ 
ready  embedded  in  their  organizations.  Other  program  offices  formalized  instructions 
while  continuing  to  coach  personnel  on  the  use  of  incentives.  Several  program  offices 
suggested  that  the  most  effective  training  is  just-in-time  training  with  the  entire 
incentive  team  attending.  They  emphasized  that  the  entire  team  includes  the  monitors, 


Training  as  a  team  is  essential  to  suttess. 


program  managers,  procuring  contracting  officers,  board  members,  fee  determin¬ 
ing  official,  and  the  administrative  contracting  officer,  if  applicable.  Some  program 
offices  suggested  that  performance  monitors  observe  how  other  program  offices 
conduct  assessments  to  help  them  appreciate  the  depth  of  evaluations.  The  interview 
surveys  clearly  lead  to  the  conclusion  that  each  program  office  must  develop  its  own 
unique,  just-in-time  incentive  training  based  on  the  incentives  in  the  acquisition  and 
the  experience  of  the  acquisition  team.  Depending  on  the  experience  of  the  team,  it 
may  be  necessary  to  obtain  assistance  from  other  program  offices  or  training  organi¬ 
zations  such  as  the  DAU.  Training  as  a  team  is  essential  to  success. 

ROLLOVER— ANOTHER  INCENTIVE  TOOL 

Unearned  award  or  incentive-based  fees  may  be  carried  forward  for  possible 
award  in  subsequent  evaluation  periods;  however,  this  process  should  be  the  excep¬ 
tion  and  not  the  rule.  The  vast  majority  of  the  program  offices  surveyed  expressed 
caution  and  noted  that  they  either  used  rollover  judiciously  or  not  at  all.  Rollover 
has  certain  disadvantages  regardless  of  how  or  when  it  is  used:  it  devalues  previ¬ 
ous  periods  and  it  allows  contractors  to  lower  performance  in  some  periods  without 
consequences  (Garrett  &  Gilbreth,  1992). 

In  certain  limited  situations,  rollover  may  be  appropriate  where  circumstances 
beyond  the  control  of  the  contractor  prevent  excellence  in  performance  or  where 
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the  government  wishes  to  motivate  excellent  performance  in  subsequent  award  fee 
periods.  An  example  of  the  latter  could  be  maximizing  the  incentive  for  the  contractor 
by  allowing  rollover  on  pre-flight  testing  when  the  successful  launch  of  a  satellite  is 
the  government’s  ultimate  goal. 


REALISTIC  INCENTIVE  STRATEGIES— PATHWAY  TO  SUCCESS 

As  described  in  the  findings,  the  research  team  visited  many  acquisition  organiza¬ 
tions  that  used  specific  techniques  to  drive  favorable  outcomes.  This  section  highlights 
several  acquisition  organizations  that  are  using  incentive  strategies  tailored  to  relate 
very  well  to  then'  desired  contractual  outcomes.  This  research  demonstrated  that  one 
size  or  type  of  incentive  strategy  does  not  fit  all  situations  or  outcomes.  While  the 
research  team  could  have  used  examples  from  many  other  acquisition  organizations,  the 
variety  and  types  of  incentives  and  outcomes  were  key  in  these  selections. 

AIR  MOBILITY  COMMAND  (AMC)  CONTRACTED  COMMERCIAL  GATEWAY 
SERVICES  AT  BALTIMORE-WASHINGTON  INTERNATIONAL  AIRPORT 

The  contract  is  a  fixed-price-award-fee  (FPAF)  services  contract  for  passenger 
handling  at  $1  million  per  year  for  5  years.  At  first  glance,  many  acquisition  profes¬ 
sionals  would  say  they  do  not  like  award  fee  on  fixed-price  contracts,  but  AMC 
thought  otherwise.  According  to  the  contracting  officer: 

We  use  an  award  fee  for  the  Commercial  Gateway  at  Baltimore 
because  this  contract  provides  a  very  customer-focused  service.  The 
contract  spells  out  the  requirements  needed;  however,  there  is  much 
more  the  contractor  can  provide  to  ease  the  stress  of  our  travelers.  We 
all  recognize  good  customer  service  when  we  see  it,  but  it  is  difficult 
to  put  in  writing.  As  various  situations  arise,  it  is  very  beneficial  to 
have  a  contractor  who  recognizes  ways  they  can  assist  and  has  the 
initiative  and  authority  to  step  up  and  do  so. 1 

The  goal  of  the  program  office  in  using  award  fee  was  to  motivate  the  contractor 
to  provide  the  military  with  the  best  commercial  air  service  possible.  The  award  fee 
earned  by  the  contractor  is  determined  at  the  completion  of  two  award  fee  periods  per 
year,  but  performance  monitors  submit  their  evaluation  reports  to  the  Functional  Direc¬ 
tor  (FD)  every  month.  Reports  fully  document  the  contractor’s  performance  during  the 
period.  An  overall  rating  of  excellent,  very  good,  good,  satisfactory,  or  unsatisfactory 
is  assessed  on  each  completed  report  to  the  FD.  In  addition  to  the  reports,  performance 
monitors  collect  all  observations  (e.g.,  customer  complaints/surveys,  etc.)  and  submit 
them  to  the  FD  every  month.  All  reports/performance  indicators  are  consolidated  by  the 
FD  into  a  performance  report  and  submitted  to  the  contracting  officer  with  a  recom¬ 
mended  rating.  These  reports  are  then  forwarded  to  the  contractor  within  20  days  after 
the  close  of  each  month.  In  making  its  recommendation,  the  Award  Fee  Review  Board 
considers  both  the  FD’s  report  as  well  as  the  contractor’s  presentation. 
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The  contractor  was  evaluated  on  four  categories:  customer  service,  passenger 
service,  baggage  service,  and  management.  The  categories  were  weighted  evenly  be¬ 
cause  the  contractor  needed  to  perform  equally  well  in  all  categories.  The  award  fee 
plan  provided  for  no  earned  award  fee  for  unsatisfactory  performance.  Additionally, 
the  plan  provided  no  award  fee  for  satisfactory  performance  (meeting  the  terms  and 
conditions  of  the  contract).  The  rationale  is  that  the  profit  inherent  in  the  fixed-priced 
contract  rewards  the  contractor  for  satisfactory  performance. 


This  fixed-prite  award  fee  tontratf  has  been  very  suuessful 
and  athieved  favorable  outtomes. 


Both  the  program  manager  and  contracting  officer  stated  that  the  award  fee 
was  very  effective  at  motivating  the  contractor,  and  this  was  validated  by  customer 
surveys.  They  also  felt  that  objective  criteria  would  not  achieve  the  level  of  favorable 
outcomes  that  they  have  achieved  with  award  fee.  This  fixed-price  award  fee  contract 
has  been  very  successful  and  achieved  favorable  outcomes.  Customer  surveys  support 
that  conclusion. 

TACOM  LIFE  CYCLE  MANAGEMENT  COMMAND,  FUTURE  COMBAT 
SYSTEMS  (FCS) 

The  FCS  is  the  Army’s  new  generation  of  manned  and  unmanned  ground  vehi¬ 
cles,  air  vehicles,  and  munitions,  each  of  which  taps  into  a  secure  network  of  superior 
combat  information.  These  weapons  systems  are  designed  to  be  a  fraction  of  the 
weight  of  current  weapons,  but  are  just  as  lethal  and  survivable. 

The  SDD  contract  is  an  innovative  CPFF  contract  with  incentive.  The  current  fee 
arrangement  includes  a  7.5  percent  fixed  fee  and  a  7.5  percent  incentive  fee  compo¬ 
nent.  The  incentive  fee  clause  in  the  contract  states: 

The  key  objectives  of  the  SDD  program  are  to  prove  out  technologies 
and  systems  integration  and  to  move  the  program  forward  into  readi¬ 
ness  for  initial  production,  at  an  affordable  cost  and  on  schedule.  The 
purpose  of  the  incentive  fee  is  to  encourage  and  motivate  the  contrac¬ 
tor  to  place  the  program  in  an  appropriate  risk  position.  To  this  end 
the  government  desires  to  incentivize  the  contractor  for  successfully 
meeting  these  objectives  within  certain  performance,  cost,  and  sched¬ 
ule  constraints.2 

The  cost  constraints  are  comprised  of  Life  Cycle  Cost  Containment  (LCCC) 
and  Average  Unit  Procurement  Cost  (AUPC).  The  incentive  fee  is  focused  on  the 
performance  of  selected  program  events.  The  SDD  contract  includes  nine  incentive 
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events  with  four  or  five  sub-criteria  defined  for  each  event.  The  events  and  sub-events 
associated  with  performance  elements  are  defined  in  the  Integrated  Master  Plan 
(IMP);  the  events  and  sub-events  pertaining  to  the  schedule  elements  are  based  on 
the  Program  Event/Milestone  thresholds  in  the  Acquisition  Program  Baseline  (APB) 
or  on  the  Integrated  Master  Schedule.  The  assessment  and  the  percentage  allocation 
of  the  available  incentive  fee  are  determined  by  mutual  agreement  of  the  government 
and  contractor,  and  are  incorporated  into  the  contract  clause  at  least  one  incentive 
event  prior  to  the  event  under  consideration.  This  continuous  dialogue  allows  the  gov¬ 
ernment  and  contractor  to  reassess  the  criteria  and  determine  if  the  weightings  still 
have  merit  as  written  or  should  be  altered,  and  if  additional  criteria  should  be  added. 
In  those  cases  where  no  incentive  fee  allocation  is  set  for  a  specific  event,  a  default 
allocation  is  applicable. 

The  incentive  categories  selected  for  the  FCS  SDD  contract  were  designed  to 
insure  that  the  program  office  could  afford  to  produce  the  items  developed  in  SDD. 
The  CPFF  with  Incentive  strategy  is  very  effective  in  motivating  the  contractor:  the 
contractor  is  meeting  cost  and  schedule  objectives;  technical  performance  is  success¬ 
fully  proceeding  as  the  first  prototype  nears  delivery;  and  the  Army  is  pleased  with 
the  contractor’s  performance. 

TACOM  LIFE  CYCLE  MANAGEMENT  COMMAND,  TOTAL  INTEGRATED 
ENGINE  REVITALIZATION  (TIGER)  PROGRAM 

The  TIGER  program  is  an  Army  initiative  to  revitalize  the  Automotive  Gas 
Turbine  (AGT)  1500  engine  fleet  that  supports  the  Abrams  Tank  and  derivative  ve¬ 
hicles — the  M1A1  tank,  M1A2  tank,  System  Enhancement  Package  (SEP)  tank,  and 
the  Heavy  Assault  Bridge  (HAB).  The  TIGER  program  will  increase  the  reliability 
of  the  AGT  1500  engine  by  improving  the  overhaul  processes  to  a  near  new  engine 
standard,  including  durability-based  design  improvements,  and  will  provide  the  sup¬ 
port  to  Anniston  Army  Depot  (ANAD)  for  the  overhaul  of  approximately  1 ,060  AGT 
1500  engines  per  year. 


The  contractor's  performance  is  measured  by  four  metrics: 
Engine  Availability,  Durability,  Cost,  and 
Small  Business  Participation. 


The  contract  is  fixed-price  with  an  incentive  fee  consisting  of  a  Program  Year 
Transition  (PYT)  and  PY  1  through  PY  3  determinations.  A  unique  aspect  of  the 
contract  is  that  the  incentive  metrics  are  objective  as  one  would  expect,  but  also 
included  therein  are  attributes  similar  to  award  fee  such  as  having  a  review  board 
and  fee  determining  official.  The  board  meets  annually  to  evaluate  and  validate  the 
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contractor’s  performance.  The  contractor’s  performance  is  measured  by  four  met¬ 
rics:  Engine  Availability,  Durability,  Cost,  and  Small  Business  Participation.  In  each 
program  year,  the  amount  of  the  fee  pool  and  the  weights  assigned  to  the  four  incen¬ 
tive  metrics  are  determined  based  on  the  events  of  the  particular  program  year.  These 
metrics  are  considered  to  be  of  key  importance  in  achieving  the  overall  performance 
goal,  which  is  to  improve  the  mean  time  between  depot  return  (MTBDR)  from  the 
current  700  hours  to  1,400  hours  at  a  reasonable  cost  by  the  end  of  PY  3. 

In  each  program  year,  the  contractor  will  be  challenged  to  exceed  the  government 
threshold.  If  contractor  performance  is  considered  satisfactory,  the  contractor  will  not 
receive  any  incentive  fee.  In  other  words,  incentive  fee  is  only  paid  for  exceeding  the 
threshold/satisfactory  performance  level. 


Using  a  fixed-price  tontratt  with  incentives  that  also  allows 
rollover...may  be  necessary  to  achieve  success. 


This  incentive  approach  is  tied  directly  to  the  metrics  creating  a  strong  link  to 
favorable  outcomes.  Additionally,  incentive  funds  not  earned  for  a  period  may  remain 
in  the  incentive  fee  pool  as  rollover.  The  program  office  is  judiciously  using  the 
rollover  incentive  funds  and  realizes  that  the  contractor  still  has  to  earn  all  incentive 
fee  payouts. 

As  can  be  seen,  this  incentive  strategy  is  a  highly  leveraged  construct  that  re¬ 
quires  the  achievement  of  many  goals  designed  to  enable  the  contractor  to  reach  the 
overall  performance  goal  of  1,400  hours  MTBDR  within  specific  cost  objectives. 
Using  a  fixed-  price  contract  with  incentives  that  also  allows  rollover  is  certainly  a 
challenging  strategy  for  the  contractor  to  meet,  but  it  may  be  necessary  to  achieve 
success.  Whether  the  contractor  will  meet  cost,  schedule,  and  performance  incentive 
parameters  is  too  early  to  tell,  but  the  Army’s  communicative  and  innovative  ap¬ 
proach  makes  success  much  more  likely. 

AERONAUTICAL  SYSTEMS  CENTER  (ASC),  F-l  5  AIRCRAFT  SUITE  6 
SOFTWARE  UPGRADE  FOR  A-E  MODELS 

The  F- 1 5  Aero  Systems  Group  uses  a  unique  CPAF  contract  to  perform  this  effort. 
While  a  typical  CPAF  contract  uses  only  subjective  criteria,  the  F-l 5  Aero  Systems 
Group  used  both  objective  and  subjective  criteria  as  they  deemed  appropriate.  They 
used  government-contractor  integrated  process  team  (IPT)  sessions  to  establish  the 
criteria  used  in  the  award  fee  periods,  which  has  been  very  successful.  Each  award  fee 
period  is  tied  to  key  deliverables  and  has  preplanned  adjustments.  In  the  first  award 
fee  period,  cost  and  schedule  were  weighted  more  heavily  than  the  other  criteria. 

Cost  and  schedule  were  each  weighted  at  30  percent,  and  program  management  and 
subcontract  management  were  each  weighted  at  20  percent.  The  subjective  evaluation 
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criteria  are  program  management  and  subcontract  management.  The  program  office 
created  a  strong  link  to  favorable  outcomes  and  reduced  program  risk  by  developing 
the  award  fee  objectives  and  linking  the  award  fee  periods  to  key  deliverables. 

It  is  interesting  to  note  that  the  contractor  initially  preferred  an  award  fee  with 
only  subjective  criteria,  and  expressed  concern  at  the  use  of  objective  criteria.  Howev¬ 
er,  when  the  program  office  structured  a  base  fee  of  3  percent,  the  contractor  accepted 
the  government’s  plan.  This  demonstrates  how  the  base  fee  can  be  leveraged,  thereby 
requiring  excellent  performance  to  receive  the  “award  portion”  of  award  fee. 


Each  of  the  four  example  programs  died  here  was  quite 
different \  but  they  used  spedfit  incentive  strategies  that 
related  very  well  to  their  desired  contractual  outcomes . 


This  CPAF  contract  is  still  in  the  early  stages  of  performance;  however,  the  use 
of  the  IPT  in  developing  the  award  fee  objectives,  linking  the  award  fee  periods  to 
key  deliverables,  and  leveraging  the  base  fee  by  only  awarding  the  award  portion  of 
award  fee  for  excellent  performance,  should  provide  a  solid  foundation  for  successful 
contract  performance. 

Each  of  the  four  example  programs  cited  here  was  quite  different,  but  they  used 
specific  incentive  strategies  that  related  very  well  to  their  desired  contractual  outcomes. 


RECOMMENDATIONS  AND  TAKE-AWAYS 
INCENTIVE  PHILOSOPHY 

Every  government  buying  activity/command  should  have  a  clearly  defined  incen¬ 
tive  philosophy  that  is  consistent  with  the  FAR,  DFARS,  and  Service/ Agency  policy. 
This  incentive  philosophy  should  be  clearly  communicated  to  all  personnel  and 
potential  contractor  sources. 

TRAINING 

The  amount  of  training  depends  on  the  team’s  experience.  Experienced  teams 
may  only  need  refresher  type  training  (e.g.,  Phase  3  below).  Inexperienced  teams 
should  consider  a  three-phased  approach  to  training.  The  first  phase  is  a  compre¬ 
hensive  core  block  of  instruction  on  incentive  contracting  for  Defense  Acquisition 
Workforce  Improvement  Act  (DAWIA)  Level  II  business  and  technical  personnel. 
This  provides  a  basic  body  of  business  and  acquisition  knowledge  for  the  individual 
acquisition  members.  The  second  phase  is  just-in-time  training  using  a  continuous 
learning  module  on  incentives.  The  third  phase  is  lessons  learned  on  a  community  of 
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practice  (COP)  or  by  classroom  teaching  from  within  or  by  an  institution  such  as  the 
DAU.  Once  the  required  amount  of  training  is  determined,  the  authors  recommend 
that  the  entire  acquisition  team  should  participate  in  Phase  3  together.  This  develops 
teamwork  in  the  learning  process,  which  benefits  the  acquisition  team. 

FEEDBACK 

Cost-plus-award-fee  and  other  incentive-type  contracts  provide  contractors  with 
numerous  avenues  to  communicate  openly  with  the  government  during  contract  per¬ 
formance.  Moreover,  incentive  contracts  provide  the  government  with  greater  input  or 
leverage  to  motivate  contractors  to  achieve  exceptional  performance.  More  specifically, 
the  authors  recommend  that  program  offices  document  in  their  contracting  file  the  use 
of  any  subjective  incentives,  rollover  features,  incentive  ratings,  creative  approaches 
to  incentivizing,  and  any  exceptions  to  policy.  Documentation  or  approvals  required  to 
take  a  certain  specialized  acquisition  approach  should  never  discourage  one  from  being 
innovative,  if  the  business  acumen  supports  such  an  approach.  The  authors  also  recom¬ 
mend  that  program  offices  create  a  glossary  of  terms,  particularly  terms  related  to  criti¬ 
cal  outcomes  or  events.  This  will  prevent  misunderstanding  between  the  program  office 
and  the  contractor  in  the  evaluation  of  incentives,  and  also  prove  helpful  for  personnel 
who  are  new  to  the  program  office  and  its  associated  terminology. 

AWARD  FEE 

When  an  acquisition  tool  comes  under  scrutiny,  the  tendency  is  to  avoid  it.  This 
has  occurred  with  award  fee  contracts.  Invariably,  in  certain  circumstances  only  a 
subjective  incentive  will  achieve  the  desired  goal  or  outcome.  However,  award  fee 
should  be  used  only  when  performance  cannot  be  determined  objectively.  The  con¬ 
tract  file  should  clearly  communicate  the  rationale  and  decision  to  use  award  fee.  This 
may  result  in  a  multiple  incentive-type  contract  containing  both  incentive  and  award 
fee  clauses.  This  is  consistent  with  new  guidance  provided  by  memorandum  from  the 
Office  of  the  Under  Secretary  of  Defense  for  Acquisition,  Technology  and  Logistics 
(OUSD[AT&L]  Memorandum,  April  2007).  The  following  recommendations  relate 
primarily  to  award  fee: 

Base  Fee.  Use  base  fee  in  CPAF  contracts.  It  provides  the  necessary  leverage 
for  the  proper  use  of  the  award  portion  of  CPAF.  The  DFARS  (216.405-2(c)  (iii)) 
currently  has  a  limit  of  3  percent  on  base  fee.  Consider  requesting  a  deviation  to  the 
DFARS  to  increase  the  base  fee  percentage  and  make  better  use  of  base  fee  as  a  lever¬ 
age  tool. 

Evaluation  Periods.  Set  the  award  fee  periods  based  on  events  or  outcomes  rather 
than  based  on  time.  This  will  ensure  that  the  award  fee  will  be  appropriate  for  the 
work  accomplished  within  the  period. 

Relate  Fees  to  Outcomes.  The  percentage  of  the  fee  pool  for  the  award  portion  of 
the  award  fee  should  be  commensurate  with  the  importance  of  the  outcomes  for  that 
award  fee  period. 

Rollover.  Rollover  should  seldom  be  used,  but  in  certain  limited  situations  the 
advantages  to  the  government  of  using  rollover  outweigh  the  disadvantages.  Weigh 
this  decision  carefully  and  always  document  the  rationale. 


146 


CONCLUSIONS 


When  it  comes  to  incentives,  evaluate  all  options.  As  shown  in  the  four  examples 
in  this  article,  one  size  does  not  fit  all  situations.  In  the  early  stages  of  acquisition 
planning,  seek  help  from  experienced  acquisition  professionals  in  the  organization 
or  from  the  DAU  faculty  who  have  experience  in  the  use  of  incentive  contracting  and 
developing  incentive  strategies.  These  experienced  professionals  can  also  be  very 
helpful  by  conducting  real-time  training  and  providing  facilitation  for  the  acquisition 
team.  Use  Defense  Contract  Management  Agency  (DCMA)  support  through  the  FAR 
delegation  process;  DCMA  can  serve  as  a  valuable  asset  for  evaluating  contractor 
performance.  Seek  cost  estimating  and  pricing  support.  In  the  acquisition  planning 
phase,  build  the  team  and  gather  market  research  information.  This  will  enable  an  ac¬ 
curate  determination  of  risk  necessary  for  determining  contract  type  and  will  help  in 
the  structuring  of  logical  and  effective  incentives  tailored  to  the  program.  Developing 
effective  and  efficient  award  and  incentive  fee  programs  is  truly  a  team  effort. 
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ENDNOTES 


1.  Commercial  Gateway  Baltimore  contracting  officer’s  E-mail  response,  January 
11,  2007,  to  research  team  interview/follow-up  question. 

2.  Incentive  Fee  Clause  exceipted  from  U.S.  Army  TACOM  Life  Cycle 
Management  Command  Contract  W56HZV-05-C-0724,  System  Development 
and  Demonstration  issued  in  2005. 
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Image  designed  by  Harambee  Dennis 


PPBE:  A  RED  OR  A  BLUE  PILL? 
CAN  DEFENSE 
SENSEMAKERS  REALLY 
BE  RATIONAL  IN  A 
HYPERTURBULENT  WORLD? 

COL  Christopher  R.  Paparone,  USA  (Ret.) 


The  author  applies  social  construction  theory  to  reveal  potential  blind 
spots  associated  with  the  technical  rationality  paradigm  rooted  in 
the  Defense  Department's  Planning,  Programming,  Budgeting  and 
Execution  (PPBE)  process.  Drawing  heavily  on  Karl  E.  Weick's  (1995) 
version  of  sensemaking  (i.e.,  using,  modifying,  rejecting,  and  creating 
new  paradigms  or  shared  mental  models  when  dealing  with  situations 
of  incoherency  and  disorderliness),  this  article  reveals  the  paradoxical 
qualities  along  three  socially  constructed  schemes:  the  Pentagon  world 
of  PPBE,  the  world  of  political  reasoning,  and  the  hyperturbulent 
world  described  as  our  contemporary  operating  environment  (COE). 
Ultimately,  the  author  argues  that  Defense  resource  management 
professionals  could  be  living  in  a  dream  world  not  unlike  that  imagined 
by  the  character,  Neo,  in  the  popular  movie,  The  Matrix©. 


“ Rationality ,  of  course,  is  a  moot  issue  when  causality  is  poorly 
understood.” 

— Russ  Marion,  The  Edge  of  Organization 


For  almost  half  a  century,  the  Department  of  Defense  (DoD)  has  enjoyed  the 
reputation  of  being  accountable  and  responsive  to  both  the  Executive  Branch 
and  Congress  by  presenting  rational  solutions  to  well-defined  requirements.  The 
DoD’s  Planning,  Programming,  Budgeting  and  Execution  process  (PPBE  for  short) 
has  evolved  into  one  of  the  most  sophisticated  and  regimented  examples  of  compre¬ 
hensive  strategic  planning  in  the  world  (Roberts,  2000;  McCaffrey  &  Jones,  2005). 
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However,  the  efficacy  of  the  strategic  planning  paradigm  as  a  method  for  allocating 
resources  (e.g.,  Gulick,  1937;  Ansoff,  1979;  Bryson,  1995;  Kaplan  &  Norton,  1996; 
Lewis,  Brown,  &  Schrader,  1999)  has  been  under  considerable  attack  for  decades 
(e.g.,  Lindblom,  1959;  Allison,  1969;  Mosher,  1969,  Rittel  &  Webber,  1973;  Stein- 
bruner,  1974;  Senge,  1990;  Mintzberg,  1994;  Kingdon,  1995;  Michael,  1997;  Stone, 
1997;  Downs,  Durant  &  Carr,  2003).  This  article  addresses  this  dichotomy  by  pre¬ 
senting  the  case  for  higher  order  sensemaking — shorthand  for  the  phenomenological 
view1  of  how  human  beings  can  purposefully  use,  modify,  reject,  create,  and  share 
paradigms  when  dealing  with  complex  or  chaotic  situations. 

This  is  an  argument  for  sensemaking,  centering  the  discussion  on  transforming 
how  to  think  about  managing  DoD  resources.  It  highlights  the  limits  of  technical 
rationality  (the  philosophical  basis  of  PPBE),  the  logic  of  political  reasoning,  and  the 
complexities  of  the  contemporary  operating  environment  (COE).  The  phenomeno¬ 
logical  proposition  is  that  by  presenting  thoughts  about  collaborative  inquiry  from  a 
social  constructionist  perspective,  DoD  professionals  (both  civil  servants  and  military 
officers)  and  their  clients  (political  appointees  or  elected  officials)  can  make  better 
shared  sense  of  managing  resources  under  complex  and  chaotic  conditions.2 


SENSEMAKING  WITHIN  THE  PPBE  CONTEXT 

The  modern  concept  of  rationality  is  a  relatively  new  one  in  the  scheme  of  world 
history.  Rene  Descartes  (1596-1650)  was  an  important  framer  of  the  enlightenment 
associated  with  the  idea  that  the  world  can  be  objectified  through  the  emerging  posi¬ 
tivist  philosophy  of  Newtonian  science.3  The  central  idea  of  scientific  (or  technical) 
rationality  is  that  objectivity  can  be  verified  with  the  content  of  a  positivistic  body  of 
knowledge  (Hacking,  1982).  Recently,  post-Newtonian  scientists  (i.e.  postpositivists) 
have  challenged  the  Cartesian  assumptions  associated  with  the  belief  in  objective 
reality — that  the  world  is  separate  from  us  and  our  conceptualizations  of  it  (e.g., 
Weick,  1995;  Hatch,  1997;  Kilduff  &  Mehra,  1997;  Whipp,  1999).  The  common 
sense  (prevalent  social-psychological  disposition)  associated  with  Rene  Descartes’ 

I  think  therefore  I  am,  is  replaced  with  the  less  common  sensemaking  premise  of,  I 
think  therefore  I  imagine. 

Sensemaking,  a  form  of  imagination,  is  characterized  by  individuals  and  groups 
using,  modifying,  rejecting,  and  creating  new  paradigms  or  mental  models  when 
dealing  with  situations  of  incoherency  and  disorderliness  (Weick,  1995).  Sensibil¬ 
ity  is  about  reaching  a  condition  of  open  receptiveness  to  emergent  and  sometimes 
counterintuitive  and  countercultural  mindfulness  as  contrasted  with  a  taken-for- 
granted  mindlessness.  The  idea  of  mindfulness  is  oriented  on  being  wholly  engaged 
in  scrutiny,  a  continuous  refinement  of  expectations  based  on  new  experiences,  and 
a  willingness  to  invent  new  expectations  (Weick,  1998).  For  example,  the  positivist- 
based  assumptions  of  strategic  planning  include  a  belief  that  predicting  pathways  to 
achieving  goals  will  bring  certain  finality  to  solving  problems.  Incommensurate  with 
that  logic,  sensemaking  implies  there  is  no  finality  because  humans  socially  construct 
a  reality  that  they  can  never  be  certain  about  (Berger  &  Luckmann,  1967;  Searle, 
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1995).  The  challenge  in  sensemaking  is  to  treat  experience  and  prior  learning  reposi¬ 
tories  (such  as  planning  habits,  doctrines,  and  rules)  as  theories  for  action  that  should 
be  tried  and  tested  continuously  in  search  of  new  mental  models  (Argyris  and  Schon, 
1978). 

This  phenomenological  approach  allows  us  to  step  back  in  a  metaphysical  way  to 
examine  the  Pentagon-created  world  of  PPBE.4  As  such,  PPBE  (based  in  the  logical 
positivism  of  operations  research  and  systems  analysis)  has  become  something  akin 
to  a  cultural  ideology  in  the  DoD;  witness  where  alternative  types  of  decision  mak¬ 
ing  are  disdainful  if  not  incomprehensible  (Paparone  &  Crupi,  2006).  This  ideology 
reflects  an  unquestioned  belief,  especially  in  numeric  values  or  metrics,  associated 
with  an  isolatable,  predictable,  and  reproducibly  testable  cause-  (ways  and  means) 
and-effect  (ends)  relationships.  The  rational  discovery  of  these  ends,  ways,  and  means 


The  tommon  sense  (prevalent  sotial-psythologital 
disposition)  assodated  with  Rene  Destartes'  I  think 
therefore  I  anv  is  replated  with  the  less  tommon 
sensemaking  premise  of,  I  think  therefore  I  imagine. 


through  technical  processes  is  believed  to  be  unbiased  by  emotions  and  unaffected 
by  unethical  political,  cultural,  and  psychological  values  that  would  otherwise  distort 
the  results  (Simon,  1997).  Although  the  process  has  evolved  to  be  a  very  compli¬ 
cated  series  of  planned  events  and  documentation,  PPBE  is  essentially  rooted  in  the 
sequential  steps  of  the  generic  rational  decision-making  process,  borrowed  from  the 
modern  scientific  method  of  hypothesis  testing:  (a)  Define  the  problem  (reducing  the 
complicated  to  a  manageable  dependent  variable);  (b)  present  all  facts  and  assump¬ 
tions  bearing  on  the  problem  (what  affects  the  variable);  (c)  develop  courses  of  action 
(COA)  to  solve  the  problem  (search  for  the  independent  variable);  (d)  select  the  best 
COA  based  on  objective  criteria  for  analyses  (how  to  make  the  independent  variable 
more  powerful  in  a  reproducible  way);  and,  (e)  implement  and  provide  feedback  (an¬ 
alyze  the  results  and  report  in  preparation  for  the  next  cycle).  This  paradigm  assumes 
problems  can  be  defined  in  relative  independence  from  other  conditions  through 
a  process  called  reductionism.  For  example,  in  the  DoD’s  force  management,  the 
current  practice  is  to  reduce  and  categorize  problems  (treated  as  dependent  variables) 
and  associate  them  with  potential  funding  of  programmatic  solutions  in  doctrine, 
organization,  training,  materiel,  leadership,  personnel,  and  facilities  (the  DoD’s  list 
of  standing  independent  variables).  The  fundamental  belief  is  that  the  outcome  of 
PPBE  serves  to  argue,  in  a  sterile  sort  of  way,  the  case  for  obtaining  and  using  public 
resources.  In  addition,  through  the  PPBE  lens,  managers  assume  that  problems  of 
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defense  are  relatively  stable  and  will  generally  be  the  same  problems  defined  now  as 
when  they  are  eventually  “solved”  five  or  more  years  from  now. 

Other  assumptions  presuppose  that  there  is  no  better  way  to  control  spending 
from  the  perspective  of  those  appointed  as  publicly  responsible  and  accountable  at  the 
top  of  the  governmental  hierarchy;  that  the  President  and  Congress  unconditionally 
expect  the  DoD  to  propose  the  most  efficient  single  course  of  action  for  spending;  and 
that  the  PPBE  approach  is  the  most  influential  way  to  obtain  and  use  resources  in  our 
system  of  government.  These  assumptions  are  as  ingrained  into  the  fabric  of  the  DoD 
culture  as  to  be  considered  by  its  members  to  be  tacit  knowledge  (Polyani,  1966). 

From  the  postpositivist  perspective,  at  least  three  issues  with  these  beliefs  are  set 
forth  in  the  PPBE  paradigm: 

■  PPBE  creates  myopic  learning.  Plans,  programs,  and  budgets  spawn  specified 
expectations;  hence,  blind  managers  who  overly  focus  on  confirming  predic¬ 
tions  rather  than  preoccupying  themselves  and  their  organizations  with  updat¬ 
ing  their  thinking  especially  in  light  of  an  uncertain  environment. 

■  PPBE  undercuts  organizational  creativity  and  improvisation.  Although  plans, 
programs,  and  budgets  seem  to  provide  some  contingent  actions  (i.e.,  plans 
for  branches  and  sequels)  based  on  present  views  of  required  capability, 
managers  shun  forms  of  adhocracy  to  deal  with  the  unexpected;  whereas, 
adhocracy  may  serve  them  and  their  clients  better  in  some  cases  than  institu¬ 
tionalized  solutions. 

■  PPBE  fosters  “mindless”  decision  traps.  Regulatory  approaches  to  budgeting 
activities  make  even  the  smartest  managers  prone  to  repeat  patterns  of  action 
that  have  worked  in  the  past  (a  form  of  mindlessness);  rather,  being  mindful  of 
the  uniqueness  of  situations  that  makes  the  pursuit  of  best  practices  or  bench¬ 
marks  seem  dangerous. 

In  contrast  to  the  mental  confines  of  strategic  planning,  continuous  sensemak¬ 
ing  demands  being  mindful,  and  both  appreciating  that  something  needs  to  be  done 
and  changing  what  to  do.  It  demands  the  recognition  that  ends,  ways,  and  means  are 
transitory  and  will  moiph  over  time  as  political  interpretations  and  environmental 
conditions  change.  It  acknowledges  that  this  process  is  interactive — the  environment 
is  affected  by  what  is  done  and  that  the  environment  will  reciprocate — in  a  never  end¬ 
ing  dynamic  of  interactive,  mutually  causal  variables  (Weick  &  Sutcliffe,  2001). 


SENSEMAKING  WITH  POLITICAL  MINDFULNESS 

The  paradigm  associated  with  political  reasoning  is  often  difficult  to  discern  from 
presumed  technically  rational  approaches  (such  as  PPBE)  because  the  process  is 
often  intentionally  or  perhaps  subconsciously  masked  by  the  appearance  of  unequivo¬ 
cal  results  of  analyses.  With  political  reasoning,  however,  there  can  be  no  set  linear 
programming  steps;  albeit,  politicians,  political  appointees,  and  their  constituents 
may  believe  or  give  the  appearance  that  they  are  one  and  the  same  (Stone,  1999).  The 
nature  of  political  reasoning  may  include  distinctive  qualities  such  as:  use  of  equivo- 
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cation;  presenting  (or  hiding)  multiple  and  convenient  interpretations  of  the  same 
rules,  policies,  and  laws;  undemocratic  forms  of  agenda  setting;  purposeful  exclusion 
of  decision  participants;  deceptive  bargaining  and  “logrolling”;  guile;  blending  alter¬ 
natives  that  appear  dichotomous;  satisfying  the  interests  of  only  the  most  powerful 
constituencies  while  trying  to  appear  to  satisfy  weaker  and  voiceless  minorities;  tak¬ 
ing  special  interest  benefits;  storytelling  to  frame  or  spin  how  the  problem  should  be 
defined;  exploiting  dogmatic  or  popular  beliefs  in  a  causal  chain  of  events;  advocating 
already  thought  of  solutions  by  purposefully  attaching  them  to  emergent  problems; 
and  using  institutionally  biased  values  that  drive  course  of  action  selection  criteria.5 


In  (ontrast  to  the  mental  tontines  of  strategit  planning ; 
tontinuous  sensemaking  demands  being  mindful \  and  both 
appredating  that  something  needs  to  be  done  and  thonging 

what  to  do. 


Implied  by  commentary  on  political  leadership,  that  “you  can  fool  some  of 
the  people. . .”  (Abraham  Lincoln,  16th  president  of  the  United  States),  when  DoD 
resource  management  professionals  who  believe  strongly  in  the  virtue  of  technical 
rationality  discover  that  the  political  form  of  reasoning  is  not  adhering  to  their  expec¬ 
tations,  they  can  become  cynical,  distrusting,  and  may  lose  their  ability  to  dialogue 
honestly  with  their  clients  (Schon,  1983).  On  the  other  hand,  when  the  political 
reasoning  process  appears  to  produce  a  desirable  outcome,  observers  who  make  sense 
of  the  world  through  the  lens  of  technical  rationality  can  also  falsely  attribute  posi¬ 
tive  results  to  policy  decisions  made  by  the  client;  albeit,  there  may  be  compelling 
evidence  that  the  outcome  was  random — simply  a  matter  of  serendipity  (Meindl, 
Ehrlich  &  Dukerich,  1984;  Kingdon,  1995).  Marion  (1999)  describes  this  attribution 
phenomenon  in  terms  of  a  ritual  where  “strategic  planning  can  provide  leadership 
with  an  opportunity  to  reinforce  its  position  in  the  pecking  order.  It  is  a  statement 
that  says  management — like  the  shaman  at  primitive  rain  dances — is  potent  and  in 
control”  (p.  219). 

With  regard  to  both  scenarios,  DoD  resource  professionals  could  view  their'  clients 
as  those  who  exploit  them,  either  for  purposes  of  blame  when  their  PPBE  analyses  are 
taken  and  appear  to  fail  or  when  the  results  of  the  PPBE  process  are  not  listened  to, 
and  yet  the  outcome  is  favorable.  Yet,  if  both  parties  were  to  agree  that  the  outcome 
was  more  random  than  predictable  with  respect  to  the  mutual  causality  inherent  to 
environmental  conditions,  then  the  ensuing  dialogue  might  be  more  open,  honest,  and 
insightful  (Schon,  1983).  Some  evidence  of  this  inability  to  plan  and  program  budgets 
over  the  long  term  can  more  readily  be  understood  using  a  chess  metaphor. 
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Episodic  strategic  planning  under  complex  conditions  is  analogous  to  trying  to 
play  chess  with  all  the  moves  planned  out  in  advance.  Modelers  of  complexity  have 
calculated  that  there  are  10 120  possible  variations  in  chess  moves  possible  in  a  single 
game.  Mitchell  Waldrop  in  his  1992  book,  Complexity,  suggests  this  “was  a  number 
so  vast  [as]  to  defy  all  metaphor.  There  haven’t  been  that  many  microseconds  since 
the  Big  Bang”  (p.  151).  He  goes  on  to  say,  “We  human  players  have  to  make  do  with 
rules  of  thumb — hard  learned  heuristic  guides  that  tell  us  what  kind  of  strategies  will 
work  best  in  a  given  situation”  (p.  151).  John  H.  Holland,  in  his  1998  book  Emer¬ 
gence,  discusses  the  extraordinary  complexity  of  chess. 

Chess  . . .  has  enough  emergent  properties  that  [it]  continues  to 
intrigue  us  and  offer  new  discoveries  after  centuries  of  study.  And  it 
is  not  just  the  sheer  number  of  possibilities.  There  are  lines  of  play 
and  regularities  that  continue  to  emerge  after  years  of  study,  enough 
so  that  a  master  of  this  century  would  handily  beat  a  master  of  the 
previous  century  (p.  23). 

The  point  is  that  chess,  with  only  a  dozen  or  so  rules,  creates  extraordinary 
complexity  that  defies  prediction.  In  much  more  complex  situations  involving  na¬ 
tional  defense,  how  can  planners  expect  to  map  strategies  when  the  “rules”  not  only 
are  difficult  to  discern,  but  which  change  dynamically  in  a  short  period  of  time?  To 
substantiate  this  doubt,  have  Defense  planners  gone  back  to  historic  plans  to  judge 
how  accurately  past  objectives  met  the  needs  of  the  future? 


Cpisodit  strategic  planning  under  complex  conditions  is 
analogous  to  trying  to  play  chess  with  all  the  moves  planned 

out  in  advance. 


The  1993  Report  of  the  Bottom-Up  Review  (the  precursor  to  the  quadrennial 
review  process)  had  only  one  force  structure  counter-terrorism  task  envisioned  during 
“peace  enforcement  and  intervention  operations.”  The  task  was  too  vague  to  tie  to  any 
specific  program  or  budget:  “securing  protected  zones  from  internal  threats,  such  as 
snipers,  terrorist  attacks,  or  sabotage”  (Aspin,  1993,  p.  47).  A  later  example  includes 
the  1997  U.S.  National  Security  Strategy  for  a  New  Century.  This  plan  had  a  sec¬ 
tion  on  transnational  threats  that  grouped  terrorism  along  with  drug  trafficking  and 
international  crime.  Counter-terrorism  goals  were  addressed  in  the  following  sentence 
(parenthetical  alphabetical  letters  represent  emphases  added): 
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Our  policy  to  counter  international  terrorists  rests  on  the  following 
principles:  (a)  make  no  concessions  to  terrorists;  (b)  bring  all  pres¬ 
sure  to  bear  on  all  state  sponsors  of  terrorism;  (c)  fully  exploit  all 
available  legal  mechanisms  to  punish  international  terrorists;  and  (d) 
help  other  governments  improve  their  capabilities  to  combat  terror¬ 
ism  (Clinton,  1997,  p.  14). 

Conspicuously  absent  in  this  historic  plan  is  the  need  to  prosecute  a  global  war 
on  teiTor  of  the  magnitude  the  United  States  is  engaged  in  today.  Finally,  the  now 
defunct  Joint  Vision  2020,  published  in  June  2000,  focused  on  a  force  protection, 
antiterrorism  goal  without  mention  of  a  major  DoD  comprehensive  role  in  combat¬ 
ing  terrorism  in  an  offensive  way.  One  could  conclude  that  these  strategy  documents 
hardly  guided  creation  and  acquisition  of  DoD  capabilities  to  counter  terrorism;  and, 
with  the  advantage  of  hindsight,  they  were  insufficiently  visionary  to  mobilize  the 
military  toward  a  global  war  on  terror  that  emerged  within  the  future  year  defense 
planning  window.  Also  important  to  note  is  that  none  of  these  documents  made 
mention  of  foreseeable  military  operations  that  would  include  the  need  for  military 
support  for  stability,  security,  transition,  and  reconstruction  operations.  The  issue  is 
not  the  quality  of  the  planning  process;  rather,  it  is  the  belief  that  the  strategy-making 
process  can  predict  the  future  with  any  reasonable  certainty.  The  entire  PPBE  process 
is  based  upon  that  belief.  Unfortunately,  environment  is  not  cooperating  with  these 
linear  expressions  of  causality. 


SHARED  MINDFULNESS  OF  THE  CONTEMPORARY 
OPERATING  ENVIRONMENT 

The  COE  is  best  described  as  hyperturbulent  and  characterized  by  the  velocity 
and  degree  to  which  patterns  of  otherwise  quasi-stable  environmental  changes  are 
shifted  to  unstable,  maladaptive  patterns  (McCann  &  Selsky,  1997).  One  convinc¬ 
ing  paradigm  that  acknowledges  the  complexity  of  sensemaking  in  the  midst  of  the 
disorder  and  chaos  associated  with  hyperturbulence  is  Rittel’s  and  Webber’s  (1973) 
theory  of  wicked  problems  (similarly  described  as  “messes”  by  Ackoff  (1999,  p. 

178).  According  to  these  observers,  “Social  problems  are  never  solved  ...  at  best  they 
are  only  re-solved — over  and  over  again”  (1973,  p.  160).  I  paraphrase  and  interpret 
Rittel’s  and  Webber’s  distinguishing  properties  of  wicked  problems  as  follows: 

■  No  definitive  formulation.  This  includes  the  recognition  that  complex  prob¬ 
lems  are  ill-defined  and/or  that  more  information  does  not  make  the  problem 
less  ambiguous. 

■  No  stopping  ride.  That  is,  past  solutions  or  best  practices  may  continue  even 
if  conditions  change,  and  the  conditions  of  the  problem  change  more  rapidly 
than  a  planned,  programmed,  or  budgeted  change  can  keep  up  with;  hence, 
the  solution  becomes  disconnected  from  the  problem  as  the  problem  morphs. 
Finally,  turnover  and  fluidity  of  participants  in  the  affected  organizations  or 
institutions  further  confound  the  process. 
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■  Not  true  or  false,  but  bad  or  good  solutions.  Solutions  are  politically,  cultur¬ 
ally,  and  psychologically  charged,  that  is,  they  are  infused  with  the  sometimes 
hidden  values  of  those  in  power  or  with  influence;  hence,  unseen  values  judg¬ 
ments  and  intuition — not  economic  reasoning — can  and  will  dominate. 

■  No  immediate  or  ultimate  test  for  unintended  consequences.  Because  the 
situation  is  so  complex,  with  variables  that  exhibit  the  dynamics  of  mutual 
causality,  no  one  or  no  group  can  predict  what  will  happen.  The  future  years 
defense  plan  approach  will  likely  be  fraught  with  “type  III  error”  (Mitroff  & 
Kilmann,  1981) — unknowingly  solving  the  wrong  problem  with  precision. 

■  May  have  one  shot  only  because  of  irreversible  consequences.  Even  if  the 
manager  acts  in  committing  resources  to  a  single  course  of  action,  the  dynam¬ 
ics  of  taking  action  itself  will  change  the  environment  and  the  previous  condi¬ 
tions  will  be  irretrievable. 

■  No  enumerable  or  exhaustive  set  of  solutions.  Courses  of  action  can  seem 
like  bad  or  worse,  or  the  lesser  of  two  evils,  or  may  even  be  incomprehensi¬ 
ble — military  planners  metaphorically  call  this  phenomenon  the  solving  world 
hunger  kind  of  impossible  challenge,  not  unlike  the  intractable  messes  associ¬ 
ated  with  prosecuting  “irregular”  warfare  with  conventional  analytical  models 
associated  with  the  military  decision-making  process. 

■  Uniqueness.  Restated,  it  is  hard  or  impossible  to  find  benchmarks  or  best 
practices  from  the  past  or  other  examples  of  success,  historic  anecdotes,  doc¬ 
trine,  or  documented  lessons  learned. 

■  Probably  a  symptom  of  another  problem.  There  is  no  single  problem  but 
a  systemic  network  of  interactive  and  interdependent  problems  that  is  too 
complex  to  unravel. 

■  Discrepancy.  The  conceived  gaps  between  ideal  end  and  where  manag¬ 
ers  perceive  things  are  can  be  explained  in  numerous  ways,  and  there  is  no 
systematic  procedure  to  get  to  the  right  answer.  This  quality  makes  Cartesian 
solutions  fruitless  but  gives  political  actors  opportunity  for  framing  a  faqade  of 
technical  rationality  to  convince  voters  to  elect  them. 

■  The  planner,  programmer,  or  budgeter  has  no  right  to  be  wrong.  Albeit, 
they  deal  constantly  with  the  reality  of  a  large,  complex  adaptive  system — or 
organized  anarchy  (Cohen,  March,  &  Olson,  1972) — that  experiences  forever 
dynamic  and  unpredictable  trajectories,  fraught  with  ambiguity,  and  complex 
causal  webs  that  defy  the  articulation  of  a  desired  end  state  or  strategic  objec¬ 
tive  (1973,  pp.  161-166). 

The  process  of  sensemaking  reveals  that  the  nature  of  the  COE  is  not  something 
managers  have  to  deal  with  as  external  to  their  daily  lives  and  the  routine  workings  of 
the  DoD.  Indeed,  managers  and  their  organizations  are  both  in  it  and  interact  within 
the  interconnected  workings  of  it  in  a  dynamic,  never-ending  way.  It  is  implausible, 
if  not  impossible,  to  isolate  the  world  of  PPBE  against  the  backdrop  of  the  hyper- 
turbulent  environment  and  the  intervening  world  of  the  political  players.  This  is 
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reminiscent  of  the  storyline  in  the  1999  movie,  The  Matrix,  where  the  main  character, 
Neo,  must  make  sense  under  paradoxical  worldviews  represented  by  a  blue  or  red  pill 
(Wachowski  &  Wachowski,  1999).  If  Neo  takes  the  blue  pill,  he  accepts  the  well-or¬ 
dered  world  to  which  he  is  accustomed,  unaware  that  he  is  asleep  and  that  his  psyche 
is  being  fed  by  a  computer  program  (called  the  Matrix).  However,  if  he  takes  the  red 
pill  it  will  mean  facing  a  deeper  meaning  of  reality  that  may  be  shockingly  unpleasant 
to  accept.  He  would  no  longer  be  isolated  from  the  harsh  reality  that  the  world  is  re¬ 
ally  a  very  messy  place,  ridden  with  surprises,  death,  and  destruction.  Neo  chose  the 
harsher,  more  uncomfortable  reality  and  ultimately  led  his  followers  to  emancipation 
through  revolution  (metaphorically,  a  transformation  that  overthrew  the  technically 
rational  world  view  provided  by  the  Matrix). 


Sodal  problems  are  never  solved.,. at  best  they  are  only  re 
solved — over  and  over  again. 


Consider  that  Defense  managers  have  created  the  conceptual  categories  of  inter¬ 
nal  and  external  environments  as  parsimonious  coping  devices  to  separate  order  (the 
“blue  pill”)  from  chaos  (the  “red  pill”).  As  Neo  struggled  with  the  seductive  stabil¬ 
ity  and  predictability  that  the  blue  pill  promised,  managers  are  culturally  enticed  to 
believe  the  DoD  can  operate  through  forms  of  technical  rationality  in  the  inner  world, 
while  not  having  to  acknowledge  the  bounded  limitations  of  technical  rationality  in 
the  outer  world  (Simon,  1997). 

Restated  in  terms  borrowed  from  the  study  of  cultures,  managers  are  emic  (ob¬ 
serving  as  an  insider)  and  not  etic  (observing  as  an  outsider)  with  respect  to  making 
sense  of  the  environment  (Martin,  2002);  hence,  they  must  perceive  themselves  to 
function  in  both  the  social-psychological  creation  of  order  from  disorder  simultane¬ 
ously  and  inseparably.  They  will  realize  that  unilateral  sensemaking  in  the  context 
framed  solely  by  the  technical  rationality  assumptions  of  the  “blue  pill  of  PPBE”  is 
a  naive  undertaking  if  they  agree  about  the  hyperturbulent  nature  of  the  COE.  The 
Marion  quote  at  the  beginning  of  this  essay  is  worth  repeating  here:  “Rationality,  of 
course,  is  a  moot  issue  when  causality  is  poorly  understood”  (1999,  p.  142). 


COLLABORATIVE,  PROFESSIONAL-CLIENT  SENSEMAKING 

Whereas  the  PPBE  is  based  in  the  idea  of  being  technically  rational  about  the 
future,  managers  contend  with  the  political  context — the  world  of  their  clients.  They 
realize  that  attempting  to  convincingly  frame  knowledge  about  the  future  for  which 
no  one  can  foretell  is  always  the  hallucination  of  linear  causality.  In  that  regard, 
PPBE  has  an  impossible  assumption  of  predictability  when  viewed  in  the  context  of 
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FIGURE  1.  A  BROADER  CONTEXT  FOR  COMPLEX  SENSEMAKING 


political  reason  in  the  midst  of  environmental  hyperturbulence.  Whereas  there  are 
no  irrefutable  assumptions  of  technical  rationality  in  the  political  context,  political 
reasoning  is  better  viewed  by  management  professionals  as  a  sensemaking  bridge 
between  the  illusion  of  predictability  framed  by  PPBE  and  the  reality  of  uncertainty 
framed  in  the  context  of  the  COE.  In  short,  their  clients  are  engaged  in  a  type  of 
reasonableness  with  the  effect  of  trying  to  imagine  something  indefinable  into  some¬ 
thing  that  is  workable.  The  more  that  savvy  resource  management  professionals  can 
work  beyond  the  context  of  the  PPBE  process,  the  more  open  they  may  be  to  sharing 
different  appreciations  with  their  clients.  As  depicted  in  the  flow  chart  shown  here, 
they  work  as  partners  with  clients  to  help  them  build  the  sensemaking  bridge — by 
“comprehending,  redressing,  constructing  meaning,  interacting  in  pursuit  of  mutual 
understanding,  and  patterning”  (Weick,  1995,  p.  6)  in  the  broader  context  of  the  COE. 

In  doing  so,  resource  management  professionals  may  also  find  new  ways  to 
think  beyond  the  misleading  sense  of  clarity  associated  with  PPBE.  They  may  have 
to  consider  the  possibility  that  PPBE  is  the  DoD  culturally  narrow  construction  of 
reality  that  serves  as  nothing  but  a  ritual  to  temporarily  bring  a  sense  of  clarity  in  the 
fog  of  chaos  (Stacey,  1992;  Dent,  1999).  For  many  managers,  a  blind  rejection  of  this 
possibility  will  prevent  a  form  of  “Cartesian  anxiety”  (Weick,  1995  citing  Bernstein, 
1983 ) — that  is,  the  avoidance  of  the  pain  and  suffering  that  would  otherwise  be 
associated  with  rejection  of  the  Newtonian  assumptions  of  the  PPBE  world.  Rather 
than  developing  cynicism  and  distrust  while  observing  the  political  sensemaking 


160 


process,  an  acceptance  of  this  possibility  will  facilitate  participating  in  the  process  as 
a  professional  sensemaker  who  appreciates  political  reasoning  and  the  complexity  of 
social  systems  in  a  global  context.  Such  managers  would  serve  to  create  knowledge 
relationships  with  clients  rather  than  helping  clients  to  enforce  a  Matrix-like  com¬ 
mand  and  control,  unidirectional,  and  superior-subordinate  relationship  inherent  to 
the  dream  world  of  PPBE. 

Indeed,  accepting  this  more  complex  way  of  shared  sensemaking  would  create  a 
risk  for  the  manager  and  a  paradox  for  the  DoD:  one  centering  on  the  idea  that  “com¬ 
mand  and  control  kills  emergence”  (McKelvey,  1999,  p.  18);  while  simultaneously 
creating  command  and  control  structures  (such  as  PPBE)  to  foster  top-level  account¬ 
ability  and  responsiveness  to  Congress  and  the  public.  The  traditional  solution  to  this 
dilemma  is  for  clients  to  use  top-down  power  to  manage  meaning.6  Fairhurst  and  Sarr 
put  it  this  way: 

The  essential  tool  of  the  manager  of  meaning  is  the  ability  to  frame. 

To  determine  the  meaning  of  a  subject  is  to  make  sense  of  it,  to 
judge  its  character  and  significance.  To  hold  the  frame  of  a  subject 
is  to  choose  one  particular  meaning  (or  set  of  meanings)  over  others. 

When  we  share  our  frames  with  others  (the  process  of  framing),  we 
manage  meaning  because  we  assert  (as  leaders)  that  our  interpreta¬ 
tions  should  be  taken  over  other  possible  interpretations  (1996,  p.  3). 

However,  if  both  managers  and  clients  embrace  the  need  for  a  collaborative  sen¬ 
semaking,  the  framing  (usually  associated  with  the  planning,  programming,  and  bud¬ 
geting  aspects  of  PPBE)  can  no  longer  be  the  sole  responsibility  of  those  appointed  at 
the  top.  Any  attempts  to  command  and  control  information  to  make  shared  sense  of  a 
COE  that  is  too  complex  for  the  technical  rationality  paradigm  to  explain  or  predict, 
may  be  perceived  by  the  enlightened  professional  as  a  form  of  propaganda — and 
reflect  a  client’s  Machiavellian  desire  for  the  subordinate  to  accept  mindlessly  their 
approved  construction  of  reality.  Top-down  framing  (dubbed  “strategic  communica¬ 
tions”  by  the  U.S.  Army)  (Eder,  2007)  force-fed  to  passive  professionals  will  instill 
cynicism.  Active  professionals  will  learn  to  operate  as  “heroes  under  a  tent,”  doing 
what  they  perceive  they  need  to  do  despite  top-down  orders  and  espoused  strate¬ 
gies  to  the  contrary  (Schon,  1983,  p.  260).  In  this  light,  the  unchallenged,  top-down 
framing  associated  with  PPBE  is  analogous  to  the  creation  of  psychic  prisons,  where 
organizational  power  is  configured  to  suppress  differences  (Morgan,  1998). 

In  the  midst  of  perceived  complexity  of  the  larger  environment,  leaders  must 
be  permitted  to  emerge  with  significantly  less  emphasis  on  formal  and  hierarchical 
appointment.  In  the  sensemaking  associated  with  the  COE  context,  involving  indeter¬ 
minism  and  mutual  causality,  the  need  for  shared  leadership  among  professionals  and 
clients  is  better  described  as  heterarchical  (i.e.,  networked)  rather  than  hierarchical 
(i.e.,  pyramidal)  (McCulloch,  1945).  Ironically,  A1  Qaeda  and  other  terrorist  networks 
seem  to  have  already  subscribed  to  this  realization  (Marion  &  Uhl-Bien,  2003). 
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TRANSFORMING  TOWARD  MORE  HOLISTIC  SENSEMAKING 


This  last  section  will  propose  what  DoD  resource  professionals  can  do  to 
deemphasize  the  paradigm  of  technical  rationality  while  highlighting  a  more  holistic 
approach  suggested  by  the  metaparadigm  of  shared  sensemaking.  Note  the  suffix, 
ing”  in  the  following  paragraph  headings  to  indicate  a  recognition  that  an  unknowable 
and  highly  complex  environment  can  be  dealt  with  only  by  rapid  cycles  of  continuous 
acting  and  learning  shared  between  professional  resource  managers  and  their  clients. 
Educat-ing,  develop-ing,  lead-ing,  communicat-ing,  and  organiz-ing  are  key  processes. 

EDUCAT-ING 

Learning  professionals  collaborate  with  their  clients.  Emphases  is  on  action 
research7  couched  in  more  effective  metaphors  (e.g.,  less  toward  mechanical  images 
and  more  toward  organic  ones)  (Morgan,  1998);  a  variety  of  mental  models  (e.g., 
“systems  thinking,”  complexity  theory,  and  competing  theories  of  the  policy  process) 
(e.g.,  Senge,  1990;  Weick  &  Sutcliffe,  2001;  Sabatier,  1999,  respectively);  and  mul¬ 
tiple  interpretive  schemes  (e.g.,  those  rooted  in  various  metaphysical  perspectives) 
(Allison,  1969;  Fisher,  1995;  Hatch,  1997;  Rajagopalan  &  Spreitzer,  1997;  Ofori- 
Dankwa  &  Julian,  2000).  The  beliefs  in  best  practices,  doctrine,  techniques,  and 
formal  procedures  falsely  convey  a  sense  of  known  cause-and-effect  relationships. 

For  example,  Schon  compares  the  philosophy  of  educating  based  in  action  research 
with  that  of  the  traditional  model  of  education  as  follows: 

Complexity,  instability,  and  uncertainty  are  not  removed  or  resolved 
by  applying  specialized  knowledge  to  well-defined  tasks.  If  anything, 
the  effective  use  of  specialized  knowledge  depends  on  a  prior  restruc¬ 
turing  of  situations  that  are  complex  and  uncertain.  An  artful  practice 
of  the  unique  case  appears  anomalous  when  professional  competence 
is  modeled  in  terms  of  application  of  established  techniques  to  recur¬ 
rent  events. . ..  It  is  difficult  for  them  to  imagine  how  to  describe  and 
teach  what  might  be  meant  by  making  sense  of  uncertainty,  perform¬ 
ing  artistically,  setting  problems,  and  choosing  among  competing 
professional  paradigms,  when  these  processes  seem  mysterious  in 
light  of  the  prevailing  model  of  professional  knowledge  (1983,  pp. 

19-20). 

In  summary,  professionals  educate  and  self-educate  for  uncertainty  and  adapta¬ 
tion  in  a  holistic  way  rather  than  using  forms  of  reductionism  inherent  to  Newto¬ 
nian  science. 

TRAIN-ING 

Professionals  recognize  that  the  clear  distinction  between  training  and  educa¬ 
tion  is  a  cultural  invention.  The  distinction  may  not  be  helpful  from  the  perspective 
that  both  should  deal  with  the  unique  cases  based  on  the  hyperturbulent  nature  of  the 
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COE.  As  with  educating,  training  should  stress  more  individual  and  group  learning 
and  shared  sensemaking  under  realistic  and  interactive,  free-playing  scenarios,  and 
less  with  scripted  exercises.  The  process  is  continuous  and  is  neither  episodic  nor 
curtailed  during  the  sensemaking  process.  Emphasize  less  the  determinism  associ¬ 
ated  with  the  task,  condition,  and  standard  model  of  success  (Roper  &  Vandergriff, 
2005).  The  notion  of  success  comes  instead  from  valuing  bricolage,  or  the  concept 
of  emphasizing  resilience  through  the  creative  use  of  existing  capabilities  by  forming 


Reiognition  that  tasks,  tonditions,  and  standards  are 
all  in  (onstant  flux  ionveys  a  mental  model  more  tlosely 
resembling  the  demands  of  the  COE. 


new  ways  to  accomplish  things  as  the  individual  or  group  learns.  Recognition  that 
tasks,  conditions,  and  standards  are  all  in  constant  flux  conveys  a  mental  model  more 
closely  resembling  the  demands  of  the  COE.  For  example,  the  metaphor  for  opera¬ 
tions  and  training  should  move  away  from  the  expectation  of  proper  “orchestration” 
associated  with  the  heuristics  of  PPBE  to  the  welcome  surprises  of  “jazz”  associated 
with  network  fluidity,  impromptu  leadership,  and  improvisation.  More  concisely, 
professionals  train  and  educate  for  uncertainty  and  value  entrepreneurial  invention 
(Weick,  1998). 

DEVELOP-ING 

A  professional-client  transformed  shared  sensemaking  should  be  oriented  on  ex¬ 
ecuting  budgets  while  exploring  ill-defined,  intractable  issues  with  an  acknowledge¬ 
ment  of  the  existence  of  wicked  problems.  In  the  COE  context,  executing  budgets 
must  be  viewed  as  a  continuous  and  collaborative  sensemaking  process  rather  than 
an  episodic  output  of  a  top-down  planning,  programming,  and  budgeting  control 
process  with  the  accompanying  over-valued  Cartesian  quest  for  prediction.  The  plan 
for  allocating  resources  should  become  a  plan-to-learn  model  under  normal  condi¬ 
tions  of  surprise  and  uncertainty  rather  than  a  plan-to-know  process  based  on  a  myth 
of  creating  certainty  and  top-down  control  (Michael,  1997).  Department  of  Defense 
resource  management  professionals  must  serve  as  the  antitheses  of  the  “self-serv¬ 
ing  elite  who  put  science-based  technique”  as  their  masquerade  of  extraordinary 
knowledge  (Schon,  1983,  p.  340).  They  learn  to  treat  their  leaders  as  clients  with 
whom  they  must  have  open  and  honest  dialogue  to  develop  sensemaking  bridges  to 
the  COE.  Through  this  partnering  for  the  purpose  of  developing  shared  sensemaking, 
the  faqade  of  technical  rationality  is  removed.  The  dialogue  may  lead  to  a  political 
acceptance  of  significantly  less  orientation  on  the  performance-based  government 
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codified  by  Cartesian  laws  and  rules  and  the  PPBE  process  (such  as  exemplified  by 
the  Government  Performance  and  Results  Act  of  1993)  (Roberts,  2000). 

Such  a  transformation  would  constitute  a  real  paradigm  shift  (Kuhn,  1996) 
toward  rewarding  exploration  and  learning  (i.e.,  “the  creation  of  knowledge  and 
meaning”)  (Kolb,  1984,  p.  52)  and  realizing  that  today’s  success  may  be  ephemeral  as 
the  environment  continues  to  be  hyperturbulent  and  as  operating  systems  go  through 
space  and  time  in  unpredictable  trajectories.  Together,  DoD  professionals  and  their 
client  community  in  short  should  dialogue  to  find  ways  to  deemphasize  the  PPB  in 
PPBE  and  be  attentive  to  learning  while  executing  to  develop  the  force  with  all  par¬ 
ticipants  engaging  in  sensemaking.  Perhaps  the  supplemental  appropriations  based  on 
the  needs  of  today  will  become  the  appropriation  methodology  of  the  future,  where 
neither  the  Congress  nor  the  President  will  expect  the  DoD  to  perform  its  rain  dance 
of  PPBE. 

LEAD-ING 

To  achieve  maximum  participation,  the  concept  of  hierarchical  authority  must 
transform  more  toward  heterarchical  leadership,  characterized  less  by  symbols  of 
rank  and  position  and  more  by  the  quality  of  sensemaking  and  ability  to  communicate 
to  others  new  ways  to  pay  attention  to  emergent  patterns  and  embrace  the  inevitabil¬ 
ity  of  surprises.  Investing  in  the  ability  of  a  heterarchical  organization  to  be  sensitive 
to  weak  signals  of  emergent  patterns  in  the  COE  is  superior  to  allocating  resources 
based  on  the  weak  attention  span  of  those  at  the  top  of  the  hierarchy  who  must  admit 


The  paradox  is  that  experiente  alone  is  no  guarantee  of 
expertness  and  that  experienced  people  may  be  trapped 
in  dysfunctional  cultural  patterns  of  repeating  what  has 
worked  in  the  past — the  tyranny  of  success  in  the  context  of 
a  hyperturbulent  environment. 


that  they  cannot  create  very  effective  hyper-adaptive  means  by  processing  cycles  of 
planning,  programming,  or  budgeting  (Schon,  1983;  Argyris,  1991).  A  prominent 
characteristic  of  complex  sensemaking  is  less  reliance  on  hierarchical  decision  mak¬ 
ing  and  more  deference  to  sharing  and  developing  expertise  in  others  (Heifetz,  1994; 
Weick  &  Sutcliffe,  2001)  and  to  those  who  are  artful  framers  of  reality  (Schon,  1983). 
The  paradox  is  that  experience  alone  is  no  guarantee  of  expertness  and  that  experi¬ 
enced  people  may  be  trapped  in  dysfunctional  cultural  patterns  of  repeating  what  has 
worked  in  the  past — the  tyranny  of  success  in  the  context  of  a  hyperturbulent  environ¬ 
ment.  Under  the  rubric  of  sensemaking,  authority  is,  by  design,  given  to  the  people 
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with  the  most  willingness  and  imaginative  potential  to  learn  to  deal  with  a  continuous 
stream  of  emergent  threats  and  opportunities. 

COMMUNICAT-ING 

Elaborate  heterarchical  communications  networks  can  help  enable  more  en¬ 
lightened  and  improvisational  forms  of  sensemaking  by  facilitating  new  sources  of 
expertise,  perhaps  outside  the  cultural  boundaries  of  the  DoD.  In  a  flexible  commu¬ 
nications  environment,  trying  to  predict  where  leadership  might  emerge  is  fruitless. 


In  a  flexible  iommunuations  environment,  trying  to  preditt 
where  leadership  might  emerge  is  fruitless . 


In  short,  organizational  communications  can  no  longer  be  considered  a  producer  of 
learnedness  and  certainty  associated  with  planned  outcomes  generated  from  the  top; 
rather,  it  is  a  never-ending  condition  of  organizing  to  share  meaning  effectively  in  a 
world  of  uncertainty,  ambiguity,  and  hyperturbulence — spawning  a  kind  of  spontane¬ 
ous  approach  to  detecting  and  unlearning  the  sensemaking  tools  that  are  not  working 
(Weick  &  Westley,  2001). 

ORGANIZ-ING 

Organizing  for  sensemaking  may  require,  as  often  as  necessary,  “search  confer¬ 
ences”  comprised  of  Defense  stakeholders  that  include  bi-partisan  members  of  the 
executive  and  legislative  branches,  along  with  others  who  can  provide  expert  knowl¬ 
edge.  The  goal  of  these  professionally  facilitated  search  conferences  is  to  gain  ap¬ 
preciation  for  the  environment  and  to  create  rapid  strategies  (Pursar  &  Cabana,  1998). 
The  outcomes  of  the  continuous  participative  process  are  statements  of  strategic 
intent  that  are  consensus-based  and  that  generate  strong  commitment  (not  to  be  con¬ 
fused  with  buy-in)  across  a  melange  of  participants.  Similar  to  the  concept  of  design 
found  in  the  recently  published  Counterinsurgency  Manual  (Department  of  the  Army, 
2006,  pp.  4-1  to  4-9),  the  idea  of  search  conferences  would  be  to  create  strategies  that 
are  swiftly  translated  into  budget  authorizations.  Participants  should  avoid  developing 
these  conferences  around  predetermined  categories  and  programmatic  established 
areas.  Instead,  they  should  organize  around  environmental  appreciation  and  topical  is¬ 
sues  that  may  reflect  emergent  networks  of  interrelated  problems  (i.e.,  messes)  (Ack- 
off,  1999).  Process  consultation  should  be  oriented  on  establishing  and  sustaining 
the  interrelationships  among  the  conference  participants  (e.g.,  Schein,  1998)  as  they 
engage  in  sensemaking.  Rank  and  position  should  be  left  at  the  door  to  help  establish 
a  climate  of  collaboration  and  collegiality.  The  overarching  value  encouraged  among 
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all  participants  is  that  we  are  all  in  this  together.  These  conferences  may  be  conducted 
virtually  with  the  use  of  electronic  communications  technologies.  The  idea  of  plan¬ 
ning  and  programming  is  transformed  to  more  flexible  forms  of  rapid,  participative, 
and  collective  learning-while-executing. 

CONCLUSIONS 

A  more  holistic  and  collaborative  approach  to  sensemaking  signals  a  DoD-wide 
looped  pattern  of  act  4 > learn  (mutual,  real  time,  interdependent  action  research 
during  execution)  rather  than  the  more  familiar  unidirectional  cause -^effect  (strategic 
planning,  programming,  budgeting  ...  and  only  then,  execution)  paradigm.  Perhaps 
a  case  study  focusing  on  how  national  laboratories  associated  with  the  U.S.  Govern¬ 
ment  can  give  insight  on  how  work  can  be  funded  without  making  comprehensive 
predictions  about  how  the  work  will  turn  out,  would  be  of  benefit.  This  article  has 
attempted  to  spur  some  thinking  about  the  potential  of  a  shared  professional-cli¬ 
ent  sensemaking  that  promotes  mindfulness  of  the  limitations  of  positivism  and  the 
PPBE  mental  model  symbolized  by  the  blue  pill  in  the  movie,  The  Matrix.  PPBE- 
style  sensemaking  is  a  form  of  mindlessness,  creating  structures  that  have  little  regard 
to  the  necessary  changing  political  interpretations  about  the  hyperturbulent  environ¬ 
ment.  The  proposition  of  more  encompassing  red  pill  models  of  educating,  training, 
force  developing,  leading  and  organizing,  and  communicating  may  help  contribute  to 
a  more  transformational  order  of  sensemaking  about  resourcing  the  force.  To  Defense 
managers,  “Wake  up.  You  have  been  living  in  a  dream  world.” 
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ENDNOTES 


1 .  The  ontological  and  epistemological  position  that  does  not  assume  reality  is 
an  objective  experience.  In  this  article,  the  author  assumes  reality  is  socially 
constructed  (Berger  &  Luckmann,  1967;  Searle,  1995). 

2.  In  philosophical  terms,  the  author  refers  to  the  “tools  of  epistemology  and 
ontology”  that  Weick  describes  sensemakers  using  as  “they  create  that  which  they 
interpret”  (1995,  p.  38). 

3.  Rutgers  (1999)  states,  “In  the  course  of  the  nineteenth  century  the  idea  of 
rationality  became  almost  exclusively  connected  with  science  and  technology, 
and  emerged  as  the  methodological  determinant  for  objectivity  and  expertise. 

In  fact,  science  became  regarded  as  the  paradigm  of  rationality.  The  scientific 
method  of  positivism  constitutes  the  strongest  expression  of  the  belief  in 
scientific  rationality.  The  founding  positivist,  Comte  (1798-1857),  believed  that 
empirical  scientific  research  can  not  only  improve  the  world  by  making  better 
humans,  but  would  enable  them  to  control  the  ravages  of  nature.  Positivism 

is  premised  on  the  idea  that  rational,  scientific  thought  can  solve  all  human 
problems  and  that  there  is  a  steady  progress  of  science  and  society.  As  a  research 
method,  positivism  goes  hand  in  glove  with  empiricism — the  belief  that  certain 
knowledge  can  only  be  arrived  at  by  means  of  observation  (ironically,  contrary 
to  rationalism).  Positivists  believed  that  it  was  a  means  to  arrive  at  objective 
knowledge.  Thus,  not  only  is  metaphysical  argument  debased  as  ‘subjective,’  but 
all  value  issues  ought  also  to  be  regarded  as  unscientific  and  thereby  out  of  the 
sphere  of  rationality”  (pp.  22-23). 

4.  What  seemingly  makes  technical  rationality  a  legitimate  paradigm  in  the  minds 
of  PPBE  sensemakers  is  what  sociologist  Max  Weber  calls  Zweckrational  or 

a  process  of  linear  reasoning  believed  to  establish  clear  means-ends  linkages. 

“If  you  do  X,  then  Y  will  happen  as  a  result”  This  perceived  and  believed  to 
be  irrefutable  legitimacy  stems  from  five  sub-beliefs:  that  an  established  legal 
code  can  claim  obedience  from  members;  that  law  is  a  system  of  abstract  rules 
that  are  applied  to  particular  cases  (and  that  application  looks  after  the  interest 
of  the  organization  within  the  limits  of  law);  that  the  person  who  exercises  this 
authority  should  obey  in  an  impersonal  manner;  that  only  in  the  capacity  of  being 
a  member  is  there  reason  to  obey  the  law;  and  that  this  obedience  is  not  attributed 
to  a  specific  person,  but  the  position  that  they  occupy.  In  short,  technical 
rationality  is  what  Weber  conveys  as  the  dominant  philosophy  of  the  archetypal 
bureaucrat  (Heydebrand,  1999). 

5.  These  reflect  a  synthesis  of  political  reasoning  from  the  works  of  Machiavelli 
(1961/1532);  Allison  (1969);  Kingdon  (1995);  Stone  (1997);  and  Zahariadis 
(1998). 
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6.  The  DoD  refers  to  this  process  of  control  as  “strategic  communications”  in  the 
lingo  of  Defense  public  affairs  specialists  (e.g.,  see  Office  of  the  Under  Secretary 
of  Defense  for  Acquisition,  Technology,  and  Logistics,  2004). 

7.  The  action  research  concept  was  developed  by  social  psychologist  Kurt  Lewin 
and  his  colleagues  in  the  late  1940s.  His  concepts  spawned  many  to  investigate 
phenomenological  aspects  of  learning,  to  include  heavily  influencing  David 
Kolb’s  culminating  work  on  experiential  learning.  Lewin’s  learning  model 
involved  human  shared  experience-reflection-abstraction-testing  cycles  that 
produce  new  and  often  profound  meaning  (Kolb,  1984,  p.  21). 
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SYSTEM  OF  SYSTEMS 
DEVELOPMENT  FOR 
THE  DoD:  TAILORING 
ACQUISITION  REFORM 
FOR  EMERGING  NEEDS 

CDR  Scott  Moran ,  USN 


When  the  Defense  Acquisition  Performance  Assessment  (DAPA)  panel 
proposed  sweeping  reforms  to  address  long-standing  problems  in  defense 
acquisition,  their  recommendations  did  not  anticipate  critical  challenges 
expected  in  the  development  of  a  System  of  Systems  (SoS).  Defense 
leaders  counting  on  revolutionary  SoS  capabilities  must  appreciate  that 
current  and  proposed  acquisition  systems  insufficiently  facilitate  SoS 
development.  This  article  describes  the  importance  of  adapting  defense 
acquisition  processes  to  enable  effective  SoS  development  and  concludes 
with  proposed  modifications  to  the  DAPA  Report  recommendations. 
Tailoring  defense  acquisition  organization,  budgeting,  and  requirements 
generation  systems  to  overcome  the  challenges  of  SoS  acquisition  will 
be  essential  for  tomorrow's  military  systems  to  realize  their  potential. 


D 


ecades  of  efforts  to  improve  defense  acquisition  processes  have  made  a  cliche 
of  the  term  “acquisition  reform.”  Given  this  history,  few  in  the  acquisition 
community  were  surprised  when  Deputy  Secretary  of  Defense  Gordon  England 
ordered  a  comprehensive  assessment  of  Department  of  Defense  (DoD)  acquisition 
processes  (England,  2005).  However,  the  expansive  scope  Secretary  England  au¬ 
thorized  for  this  review  signaled  a  desire  to  examine  the  problem  more  holistically. 
His  request  for  an  approach  examining  “every  aspect”  of  acquisition  contrasted  with 
many  previous  reform  efforts  that  focused  primarily  on  internal  defense  acquisition 
community  issues. 

Authorizing  such  a  comprehensive  approach  acknowledged  a  past  tendency  to 
underestimate  the  effects  of  complex  interactions  among  acquisition  personnel  and 
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other  stakeholders  in  requirements  generation,  oversight,  and  financial  management 
roles.  The  panel  answering  Secretary  England’s  request  met  his  challenge,  provid¬ 
ing  an  all-encompassing  assessment  of  defense  acquisition  and  proposing  sweeping 
systemic  reforms  in  its  January  2006  Defense  Acquisition  Performance  Assessment 
(DAPA)  Report.  If  implemented,  the  DAPA  Report's  key  recommendations  will 
fundamentally  alter  the  DoD  acquisition  framework. 

Given  many  current  defense  acquisition  problems,  it  was  natural  for  the  DAPA 
assessment  panel  to  focus  primarily  on  observations  within  today’s  environment. 
However,  plans  for  defense  transformation  portend  difficulties  that  may  significantly 
aggravate  the  acquisition  problems  experienced  within  DoD.  In  particular,  plans  to 
leverage  technology  to  build  flexible  and  integrated  “systems  of  systems”  will  prove 
especially  challenging. 


Plans  to  leverage  technology  to  build  flexible  and  integrated 
"systems  of  systems"  will  prove  espedally  challenging. 


This  article  examines  some  misconceptions  associated  with  system  of  systems 
(SoS)  acquisition  and  proposes  refinements  to  tailor  the  DAPA  Report  recommenda¬ 
tions  to  better  address  these  challenges.  To  do  this,  it  briefly  describes  the  defense 
acquisition  framework  to  place  reform  efforts  in  context;  and  then  shows  how  DoD 
plans  demonstrate  the  intent  to  rely  on  advanced  technological  solutions,  such  as 
system  of  systems.  This  approach,  while  conceptually  appealing,  presents  significant 
problems  to  the  acquisition  community. 

One  of  these  hurdles  is  the  need  to  accept  a  more  realistic  viewpoint  regarding 
the  maturity  of  System  of  Systems  Engineering  (SoSE).  The  Defense  Acquisition 
Guidebook  (DAG,  2006,  p.  100)  begins  a  definition  of  SoSE  as  “planning,  analyzing, 
organizing,  and  integrating  the  capabilities  of  a  mix  of  existing  and  new  systems  into 
an  SoS  capability  greater  than  the  sum  of  the  capabilities  of  the  constituent  parts.” 

The  DAG  description  of  SoSE  goes  on  to  acknowledge  the  inherent  complexities  of 
this  emerging  discipline.1  However,  because  traditional  tools  such  as  Systems  Engi¬ 
neering  (SE)  have  been  monumentally  successful  in  defense  acquisition,  there  is  a 
strong  belief  that  SE  is  easily  adaptable  to  SoS  development.  While  SE  has  proven  in¬ 
dispensable  for  development  of  enormously  complex  systems,  the  emerging  demands 
of  SoSE  will  likely  outstrip  SE’s  utility. 

When  the  DAPA  Report  and  the  challenges  of  SoS  development  are  considered 
together,  it  becomes  clear  that  the  DAPA  panel’s  recommendations  only  partially 
address  concerns  that  complicate  SoS  development.  To  facilitate  SoS  acquisition 
for  defense  needs,  this  article  concludes  with  suggestions  to  better  align  the  DAPA 
Report  recommendations  with  DoD  plans. 
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THE  DEFENSE  ACQUISITION  FRAMEWORK: 
UNDERSTANDING  THE  SCOPE  OF  REFORM 


Determining  exactly  what  is  being  discussed  assumes  particular  importance  in 
defense  acquisition  because — depending  on  context  and  audience — the  term  “acquisi¬ 
tion”  is  used  to  describe  different  constructs.  Particularly  important  for  this  paper  are 
two  distinct  perspectives  that  are  commonly  distinguished  by  the  terms  “little  a”  and 
“big  A”  acquisition.2 

“Little  a ”  vs.  “Big  A”:  The  Defense  Acquisition  Framework  Through  Two  Lenses 

“Little  a”  acquisition  describes  the  activities  that  occur  within  the  Defense  Ac¬ 
quisition  System  (DAS).  Typically,  factors  such  as  cost,  schedule,  performance,  and 
risk  concern  DAS  participants  as  they  work  to  develop  actual  systems.  However,  two 
additional  and  distinct  decision  support  systems  specify  warfighter  requirements  and 
provide  funding. 

The  Joint  Capabilities  Integration  and  Development  System  (JCIDS)  is  used 
to  define  and  generate  requirements.  Resources  are  managed  through  yet  another 
process  that  coordinates  cost  estimating  and  funding  activities,  known  as  Planning, 
Programming,  Budgeting  and  Execution  (PPBE).  The  triumvirate  of  DAS,  JCIDS, 
and  PPBE  is  intended  to  establish  an  integrated  defense  acquisition,  technology,  and 
logistics  (AT&L)  life  cycle  management  framework  (IFC,  2005).  The  term  “big  A” 
usually  refers  to  this  larger  framework  of  three  interconnected  and  interlinked  acqui¬ 
sition  systems  (Figure  1). 

Observers  of  defense  acquisition  processes  increasingly  emphasize  the  impor¬ 
tance  of  understanding  problems  that  arise  from  interrelationships,  interdependencies, 
and  conflicts  among  these  three  systems.  The  need  to  harmonize  different  perspec¬ 
tives,  interests,  and  objectives  across  the  framework  presents  a  fundamental  defense 


FIGURE  1.  THREE  INTERACTING  SYSTEMS  IN  "BIG-A"  ACQUISITION 


DOD  Decision  Support  Systems 


Effecive  Interaction  is  Essential 
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acquisition  challenge.  However,  critics  sometimes  question  the  effectiveness  and 
feasibility  of  such  cooperation.  The  DAPA  Report  (2006)  focuses  on  this  very  point, 
characterizing  current  relationships  as  uncoordinated,  fragmented,  disconnected, 
and  unstable.  Furthermore,  the  DAPA  Report  paints  a  picture  of  even  more  complex 
stakeholder  interactions,  expanding  the  notion  of  “big  A”  to  include  the  acquisition 
workforce,  organizations,  and  industry  (Figure  2). 


FIGURE  2.  DAPA  PANEL  VIEW  OF  ACQUISITION  FRAMEWORK 


There  are  fundamental  disconnects  in  DoD  management  systems  and  Congressional  oversight  driven  by 
competing  value  and  objectives  that  create  government-induced  instability  in  our  acquisitional  programs 
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DEFENSE  ACQUISITION:  A  SYSTEM  UNDER  CONTINUAL  REFORM 


Previous  observers  have  made  similar  observations  of  dysfunction  in  the  “big 
A”  framework.  Notably,  the  landmark  “Culture  Report”  (GAO,  1992)  highlighted 
deep  conflicts  among  the  three  decision  support  systems.  However,  these  structural 
problems  were  overlooked  in  subsequent  reform  efforts  in  favor  of  more  palatable, 
incremental  changes. 

The  futility  of  such  small-step  acquisition  reform  has  become  increasingly  appar¬ 
ent.  The  DAPA  Report  (2006)  documents  seven  successive  acquisition  reform  initia¬ 
tives  attempted  since  1992.  Most  of  these  focused  on  process  improvement  within  the 
DAS,  rather  than  structural  changes.  Thus,  despite  well-intentioned  goals  of  reducing 
costs,  accelerating  schedules,  and  better  defining  requirements,  previous  reforms  have 
produced  limited  results. 
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THE  DAPA  PANEL:  A  BROAD  MANDATE  ENABLES  A  BOLD  APPROACH 


The  comprehensive  assessment  that  Secretary  England  requested  from  the  DAPA 
panel  offers  an  opportunity  to  break  this  cycle.  Opening  the  examination  to  all  aspects 
of  acquisition,  the  three  decision  support  systems  were  considered  as  an  interacting 
system.  Reporting  from  this  broad  perspective,  many  DAPA  Report  (2006)  findings 
echo  similar  observations  to  those  found  in  GAO’s  Culture  Report. 

As  noted  in  the  Culture  Report,  the  DAPA  panel  uncovered  incompatibilities  in 
behavior  that  emerged  from  divergent  organizational  values.  Differences  in  values 
stem  from  inconsistencies  among  governing  instructions,  driving  factors  (i.e.,  need- 
based,  calendar-based,  or  event-based),  stakeholder  interests,  and  distinctions  in  orga¬ 
nizational  cultures.  Observing  these  problems,  the  DAPA  panel’s  findings  criticize  the 
most  fundamental  elements  of  acquisition:  organization,  workforce,  budget,  require¬ 
ments,  acquisition  processes,  and  industry’s  role. 

Several  of  the  recommendations  from  the  DAPA  Report  (2006)  are  discussed  in 
a  later  section  of  this  article.  However,  a  major  finding  of  the  DAPA  panel  sets  the 
stage  for  a  discussion  of  DoD  plans  to  employ  systems  of  systems  in  the  future:  “stra¬ 
tegic  technology  exploitation  is  a  key  factor  that  allows  the  U.S.  to  maintain  dominant 
military  capabilities”  (p.  7).  This  finding  is  consistent  with  overarching  guidance  for 
defense  transformation  as  well  as  multiple  roadmaps  that  establish  systems  of  sys¬ 
tems  as  fundamental  building  blocks  of  tomorrow’s  forces. 


ADVANCED  TECHNOLOGY  FOR  DoD  TRANSFORMATION: 

THE  RELEVANCE  OF  SoS  DEVELOPMENT 

Improved  U.S.  military  capabilities  are  invariably  linked  to  advanced  technology. 
Planning  documents  reveal  high  expectations  for  concepts  such  as  transformation, 
network-centric  operations,  and  adaptive  systems  of  systems.  However,  bringing 
these  concepts  to  fruition  presents  daunting  challenges  to  an  acquisition  system  that 
strains  to  provide  affordable  and  timely  capabilities  to  warfighters  today. 

DEFENSE  TRANSFORMATION:  CONTINUED  STRESS  ON  THE 
ACQUISITION  SYSTEM 

Commitment  to  leveraging  advanced  technology  is  signaled  throughout  the 
defense  establishment.  The  Secretary  of  Defense  establishes  networking  as  a  corner¬ 
stone  of  military  transformation  in  the  National  Defense  Strategy  by  stating,  “The 
foundation  of  our  operations  proceeds  from  a  simple  proposition:  the  whole  of  an 
integrated  and  networked  force  is  far  more  capable  than  the  sum  of  its  parts”  (OSD, 
2005,  p.  14).  Uniformed  leaders  echo  this  approach.  The  National  Military  Strategy 
provides  a  typical  example,  describing  the  desired  Joint  Force  as  “fully  integrated, 
networked,  decentralized,  [and]  adaptable”  (CJCS,  2004,  p.  15). 

Furthermore,  Service  plans  flesh  out  these  concepts,  attributing  robust  capabilities 
to  fully  networked,  interoperable,  and  adaptive  systems  of  systems.  The  Air  Force’s 
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vision  provides  a  highly  optimistic  example:  “With  advanced  integrated  aerospace 
capabilities,  networked  into  a  system  of  systems,  we’ll  provide  the  ability  to  find,  fix, 
assess,  track,  target,  and  engage  anything  of  military  significance  anywhere”  (USAF, 

2002,  p.  6). 

Beyond  providing  state-of-the-art  military  capabilities,  planners  also  envision 
future  meta-systems  as  completely  and  flawlessly  interoperable.  One  example  is 
found  in  the  USAF  Transformation  Flight  Plan  (2004,  p.  69),  which  promises  “a  joint 
fire  control  system  of  systems  that  enables  the  Joint  Force  Commander  to  seamlessly 
across  the  sensor-to-shooter  assets  of  all  the  Services,  put  a  cursor  over  a  target  in  a 
timely  manner.” 


The  difficulties  experienced  to  date  highlight  the  need  to 
reconcile  substantially  differing  opinions . 


Conceptually,  the  acquisition  community  has  also  embraced  SoS  development  as 
essential  for  meeting  future  military  needs.  Commitment  to  this  principle  is  reflected 
in  guidance  that  links  success  in  defense  transformation  to  “network-centric  opera¬ 
tions  and  on  individually  complex  systems  linked  together  in  complex  systems-of- 
systems”  (DAG,  2006,  pp.  170-171).  Yet  initial  systems  of  systems  development 
efforts  suggest  the  need  for  new  methodologies  (Zenishek  &  Usechak,  2005;  Brown 
&  Flowe,  2005;  Luman  &  Scotti,  1996).  The  difficulties  experienced  to  date  high¬ 
light  the  need  to  reconcile  substantially  differing  opinions  regarding  the  distinctions 
between  an  SoS  and  a  highly  complex  system. 


CONFLICTING  APPROACHES  TO  SOS  DEVELOPMENT 

Capabilities  projected  for  the  technologically  superior  force  of  the  future  require 
interoperability  that  eclipses  the  state-of-the-art.  Descriptions  of  future  meta-systems 
imply  astonishing  utility  and  flexibility  with  promises  of  “multiple  autonomous  embed¬ 
ded  complex  systems  that  can  be  diverse  in  technology,  context,  operation,  geography, 
and  conceptual  frame”  (Keating  et  ah,  2003,  p.  41).  Most  important,  transformation 
proponents  envision  these  interlinked  meta-systems  providing  capabilities  far  exceed¬ 
ing  those  of  their  individual  components.  This  important  aspect  of  an  SoS — exponen¬ 
tially  complementary  capabilities — is  the  basis  of  proposals  to  develop  affordable  yet 
tremendously  capable  military  forces  centered  on  adaptive  systems  of  systems. 

Many  voices  in  the  defense  acquisition  community  assert  that  the  discipline  of 
Systems  Engineering  is  adequate  for  SoS  development.  However,  findings  of  re- 
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searchers  who  study  System  of  Systems  Engineering  raise  serious  questions  about 
this  position.  These  experts  believe  that  the  immaturity  of  SoSE  warrants  reexamina¬ 
tion  of  basic  assumptions  regarding  the  ability  of  SE  to  underpin  SoS  development. 

SYSTEMS  ENGINEERING:  UTILITY  AND  FUTURE  APPLICABILITY  TO  AN  SOS 

Because  the  discipline  of  Systems  Engineering  is  central  to  defense  acquisition, 
adherents  are  enthusiastic  about  its  continued  validity  for  SoSE.  Deep  commitment  to 
SE  principles  is  reflected  in  high-level  direction  that  requires  all  acquisition  programs 
to  be  “managed  through  the  application  of  a  systems  engineering  approach”  (OSD, 
2003,  p.  7).  The  Defense  Acquisition  Guidebook  (DAG,  2006)  fleshes  out  this  man¬ 
date,  providing  a  chapter  dedicated  entirely  to  SE. 

In  its  SE  chapter,  the  DAG  (2006)  recommends  a  “robust  systems  engineering 
approach”  for  all  programs  “regardless  of  acquisition  category”  (p.  21).  To  facilitate 
this,  the  DAG  details  SE’s  best  practices,  providing  guidance  on  technical  and  man¬ 
agement  processes  to  be  applied  throughout  a  program’s  life  cycle.  This  wholesale 
confidence  results  from  decades  of  successful  application  of  SE  principles  in  develop¬ 
ing  systems  of  staggering  complexity. 

Having  demonstrated  enormous  utility  for  complex  development  efforts,  SE 
seems  ideally  suited  for  the  interconnected  systems  envisioned  for  the  future.  Thus, 
the  DAG’ s  (2006)  recommendation  of  SE  for  SoS  development  is  unsurprising: 
“Systems  of  systems  should  be  treated  and  managed  as  a  system  in  their  own  right, 
and  should  therefore  be  subject  to  the  same  systems  engineering  processes  and  best 
practices  as  applied  to  individual  systems,”  (p.  100). 

The  Under  Secretary  of  Defense  for  Acquisition,  Technology  and  Logistics 
(USD[AT&LJ)  underscores  this  guidance  in  a  policy  memo  specifically  addressing 
SE’s  central  role  for  SoSE  efforts: 

Application  of  rigorous  systems  engineering  discipline  is  para¬ 
mount.  . ..  This  is  especially  true  as  we  strive  to  integrate  increasingly 
complex  systems  in  a  family-of-systems,  system-of-systems  context. 

Systems  engineering  provides  the  integrating  technical  processes 
to  define  and  balance  system  performance,  cost,  schedule,  and  risk 
(Wynne,  2004). 

However,  the  Defense  Department’s  ability  to  apply  SE  to  complex  systems  has 
come  under  scrutiny  in  recent  studies  (DoD  JDCST,  2004;  DAPA  Report,  2006).  The 
DAG  (2006)  acknowledges  some  unique  SoS  development  challenges,  calling  atten¬ 
tion  to  factors  such  as  “greater  complexity  of  integration  efforts”  and  “engineering 
under  the  condition  of  uncertainty”  (p.  100).  While  these  viewpoints  reveal  a  budding 
understanding  of  the  inherent  difficulties  in  SoS  development,  the  challenges  associ¬ 
ated  with  SoS  implementation  may  be  more  daunting  than  currently  appreciated  by 
acquisition  professionals. 
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SYSTEM  OF  SYSTEMS  ENGINEERING:  BEYOND  ADAPTATION  OF 
TRADITIONAL  SE 


Stakeholders  in  government  and  academia  have  begun  to  recognize  fundamental 
differences  between  the  disciplines  of  SE  and  SoSE.  To  develop  and  apply  appropri¬ 
ate  SoSE  methodologies  for  the  DoD,  a  System  of  Systems  Engineering  Center  of 
Excellence  (SoSECE)  has  been  established  under  the  auspices  of  the  USD(AT&L). 
This  organization  has  coordinated  several  SoSE  conferences  as  has  the  Institute  of 
Electrical  and  Electronics  Engineers  (IEEE).  However,  the  distinctions  between  SE 
and  SoSE  are  not  yet  widely  appreciated.  Experts  at  the  National  Center  for  Systems 
of  Systems  Engineering  describe  some  of  these  differences  in  a  seminal  paper  that 
characterizes  SoSE  maturity  “in  the  embryonic  stages  of  development”  (Keating  et 
al.,  2003,  p.  36).  To  frame  a  discussion  of  the  dissimilarities  between  SE  and  SoSE, 
these  authors  consider  eight  significant  areas  of  distinction:  Focus,  Objective,  Ap¬ 
proach,  Expectation,  Problem,  Analysis,  Goals,  and  Boundaries. 


Dedsions  intended  to  alleviate  one  problem  often  tarry 
unintended  tonsequentes  that  aggravate  others. 


Many  of  these  differences  apply  directly  to  SoS  acquisition  in  the  DoD.  Keating 
et  al.  (2003)  argue  that  SoSE  requires  a  fundamental  shift  in  focus  from  the  devel¬ 
opment  of  individual  systems  to  the  integration  of  multiple  complex  systems.  Fur¬ 
thermore,  Keating  et  al.  describe  the  importance  of  methodology-based  rather  than 
process-based  approaches  for  SoSE.  While  DoD  has  made  significant  progress  in 
improving  interoperability,  current  organizational  structures  are  optimized  for  acqui¬ 
sition  of  single — albeit  highly  complex — systems  through  process-based  approaches. 
Shifting  cultural  norms  to  reflect  a  focus  and  approach  compatible  with  SoS  develop¬ 
ment  would  suggest  significant  organizational  adaptation. 

Similar  changes  will  be  required  to  efficiently  budget  for  SoS  development  ef¬ 
forts.  Keating  et  al.  (2003)  demonstrate  the  importance  of  flexible  system  boundaries 
and  pluralistic  system  goals  in  SoSE.  However,  these  principles  contrast  sharply  with 
current  practices  and  standards  that  segregate  acquisition  funds  by  solid  boundaries 
and  unitary  program  goals.  Thus,  adapting  the  PPBE  system  to  cope  with  these  SoS 
characteristics  will  become  increasingly  important. 

Generating  appropriate  requirements  for  an  SoS  will  also  require  a  different 
approach.  The  JCIDS  operates  most  effectively  when  problems  are  clearly  defined, 
expected  capabilities  are  linked  to  system  characteristics,  and  overarching  objec¬ 
tives  can  be  met  by  optimizing  performance  within  cost  and  schedule  guidelines.  SE 
provides  an  ideal  analytical  framework  for  this  JCIDS  construct.  In  contrast,  effective 
SoS  development  requires  a  radically  different  requirements  paradigm  that  describes 
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“emergent”  rather  than  defined  behaviors,  provides  “satisfying”  rather  than  optimiz¬ 
ing  criteria,  and  defines  “initial  responses”  rather  than  final  solutions  (Keating  et  al., 
2003,  p.  40).  Therefore,  requirements  generation  will  require  significant  cultural  and 
procedural  changes  to  accommodate  SoS  development. 

Although  not  primarily  focused  on  engineering  issues,  the  DAPA  panel  expressed 
pessimism  regarding  the  readiness  and  competence  of  DoD  systems  engineering  to 
support  “large-scale  integration  efforts”  ( DAPA  Report,  2006,  p.  29).  The  panel’s 
recommendations  address  this  shortfall  and  would  significantly  improve  the  defense 
acquisition  community’s  ability  to  adapt  to  many  future  needs.  However,  tailoring 
these  recommendations  to  address  SoS  acquisition  could  substantially  improve  DoD’s 
ability  to  meet  its  stated  goals. 


DAPA  FINDINGS  AND  RECOMMENDATIONS: 

ADAPTING  REFORM  FOR  SOS  ACQUISITION 

The  DAPA  Panel  returned  many  recommendations  addressing  current  acquisi¬ 
tion  shortfalls.  However,  in  three  areas — Organization,  Budget,  and  Requirements 
Generation — the  DAPA  Report  (2006)  recommendations  could  be  adapted  to  better 
prepare  the  DoD  for  SoS  acquisition. 

ORGANIZATION:  HEEDING  THE  DAPA  PANEL'S  CALL  TO  BREAK 
STOVEPIPES  AND  BARRIERS 

In  its  broad  review,  the  DAPA  Panel  identifies  organizational  barriers  that  hin¬ 
der  efficient  program  execution.  Built  around  “highly  complex”  and  “fragmented” 
mechanisms,  the  acquisition  framework  produces  deeply  entrenched  and  destructive 
instabilities  ( DAPA  Report,  2006,  p.  4).  Furthermore,  organizational  problems  extend 
to  the  highest  levels  of  DoD  bureaucracy;  the  DAPA  Panel  notes  that  fragmentation  is 
institutionalized  by  not  connecting  “budget,  acquisition,  and  requirements  processes 
...  at  any  level  below  the  Deputy  Secretary  of  Defense”  (p.  24).  Effectively  segre¬ 
gated  into  discrete  communities,  stakeholders  act  without  appreciating  consequences 
in  a  broader  organizational  context.  As  a  result,  decisions  intended  to  alleviate  one 
problem  often  carry  unintended  consequences  that  aggravate  others. 

DAPA’s  Recommendation  on  Organization.  The  DAPA  panel  addresses  dysfunc¬ 
tional  relationships  by  proposing  a  fundamental  reorganization  to  the  “big  A”  frame¬ 
work.  The  DAPA  Report  (2006)  recommends  that  each  Service  establish  a  four-star- 
Acquisition  Systems  Command  (ASC)  to  consolidate  and  integrate  budget,  acquisi¬ 
tion,  requirements  personnel,  and  responsibilities.  However,  this  approach  might 
complicate  efforts  to  coordinate  acquisition  of  SoS  programs  with  joint  participation. 

Organizational  Problems  Specific  to  SoS  Acquisition.  Consolidation  of  acquisi¬ 
tion  efforts  within  ASCs  would  represent  an  important  step  in  overcoming  cultural 
barriers.  However,  the  proposal  to  vest  this  power  in  Service  systems  commands 
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would  retain — and  perhaps  raise — barriers  that  separate  individual  Services.  These 
barriers  could  present  especially  thorny  problems  for  program  managers  (PMs)  at¬ 
tempting  to  integrate  SoS  components  across  Service  lines. 

Achieving  SoS  interoperability  requirements  will  require  extensive  ongoing 
coordination  among  developers  of  each  component  system.  This  coordination  will  only 
be  possible  within  an  organizational  structure  where  a  wide  spectrum  of  cost,  schedule, 
risk,  and  requirements  trade-offs  between  component  programs  can  be  resolved.  Weigh¬ 
ing  appropriate  tradeoffs  will  be  especially  difficult  when  developers  face  the  tremen¬ 
dously  difficult — and  poorly  understood — challenge  of  integrating  systems  in  various 
stages  of  maturity  into  a  SoS.  Current  and  proposed  organizational  systems  simply  lack 
the  mechanisms  to  effectively  and  objectively  carry  out  such  complex  assessments. 

To  capitalize  on  the  promise  of  technological  superiority  offered  by  an  SoS, 
developers  must  work  within  an  organizational  construct  that  fosters  the  emergence 
of  complementary  capabilities.  The  DAPA  panel’s  recommendation  to  consolidate 
development,  budgeting,  and  requirements  functions  could  represent  an  important 


Consolidating  SoS  development  efforts  in  one  organizational 
strurture  would  offer  the  potential  to  perform  limited  budget 
trade-offs  among  tomponent  systems  as  priorities  (hanged. 


first  step  toward  streamlining  acquisition  efforts,  but  inter-Service  tensions  in  joint 
SoS  programs  will  still  exist.  While  PMs  are  certainly  encouraged  to  perform  this 
coordination  independently,  they  simply  lack  the  incentives,  time,  and  information  to 
do  so  effectively. 

In  some  cases,  Joint  Program  Offices  (JPOs)  could  provide  oversight  for  such 
coordination.  However,  JPOs  incur  substantial  bureaucratic  burdens.  As  systems  of 
systems  proliferate  in  coming  years,  more  efficient,  streamlined,  and  effective  organi¬ 
zational  models  for  joint  SoS  development  will  become  essential. 

The  organizational  structure  proposed  by  the  DAPA  Report  (2006)  should  be 
amended  to  clearly  delineate  mechanisms  to  facilitate  inter-Service  cooperation  in 
programs  with  extensive  joint  equities,  such  as  an  SoS.  One  such  model  would  assign 
SoS  sponsorship  to  a  lead  Service.  Today,  acquisition  sponsors  assume  responsibil¬ 
ity  for  “all  common  documentation,  periodic  reporting,  and  funding  actions”  (CJCS, 
2005,  A-6).  Consolidation  (and  expansion)  of  these  functions  for  an  SoS  could 
overcome  many  coordination  problems  if  a  Program  Executive  Officer  (PEO)  or  PM 
were  appropriately  empowered.  Because  this  approach  might  be  controversial  within 
the  DAPA  panel’s  proposed  Service-based  ASC  organizational  structure,  provisions 
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for  leadership  and  sponsorship  of  joint  SoS  development  efforts  should  be  clearly 
articulated  in  acquisition  restructuring  plans. 

BUDGET:  EMPOWERING  THE  SPONSOR  OF  A  SYSTEM  OF  SYSTEMS 

Consolidating  SoS  development  efforts  in  one  organizational  structure  would 
offer  the  potential  to  perform  limited  budget  trade-offs  among  component  systems 
as  priorities  changed.  However,  implementing  budget  changes  that  could  support 
an  integrated  plan  to  develop  and  introduce  SoS  capabilities  would  require  more 
disciplined  and  flexible  resource  management  than  is  typically  demonstrated  in  DoD 
acquisition  programs. 

The  DAPA  Report  (2006)  reaffirms  the  existence  of  fiscal  challenges  described  in 
many  prior  defense  acquisition  studies.  However,  most  previous  surveys  focused  on 
“little  a”  acquisition  issues.  Thus,  while  many  prior  criticisms  addressed  legitimate 
problems,  some  of  these  were  symptoms  of  systemic  flaws  in  the  “big  A”  framework. 
The  DAPA  Report  (2006)  employs  a  wider  scope  to  describe  how  interactions  of  the 
three  defense  acquisition  decision  support  systems  destabilize  budgets. 

DAPA’s  Recommendation  on  Budget  Stability.  To  improve  budget  stability,  the 
DAPA  Report  (2006)  recommends  establishing  a  Stable  Program  Funding  Account 
(SPFA)  for  Acquisition  Category  I  (ACAT I)  programs.  Programs  in  an  SPFA  would  be 
protected  against  the  cascade  of  unintended  consequences  flowing  from  budget  repriori¬ 
tizations.  However,  providing  such  a  buffer  would  depart  dramatically  from  traditional 
resource  management  processes  and  significantly  alter  entrenched  power  relationships. 

Acknowledging  the  political  difficulties  of  instituting  such  fundamental  change 
in  budgeting,  the  DAPA  Report  nonetheless  focuses  its  SPFA  recommendation  on 
programs  with  the  largest  capital  expenditures.  While  the  Quadrennial  Defense 
Review  Report  offers  strong  support  for  the  SPFA  concept,  Congress  may  be  reluctant 
to  forego  its  traditional  oversight  of  ACAT  I  programs  (OSD,  2006).  Instead,  select¬ 
ing  a  “test-bed”  program  that  would  derive  great  benefits  from  budget  stability  might 
provide  a  more  politically  realistic  strategy  to  introduce  SPFAs  and  generate  congres¬ 
sional  confidence  in  the  concept.  An  SoS  development  effort  offers  great  potential 
for  such  an  initiative  because  the  shifting  boundaries  and  goals  in  SoS  component 
systems  will  tend  to  be  especially  destabilizing  to  their  budgets. 

Budget  Problems  Specific  to  SOS  Development.  The  current  defense  acquisition 
system  typically  rewards  managers  capable  of  obligating  and  expending  resources 
that  were  planned  years  in  advance.  Changes  in  resource  requirements,  either  above 
or  below  the  appropriated  level,  are  viewed  as  problems  and  quickly  attract  unwel¬ 
come  scrutiny.  PMs  of  SoS  components  will  face  aggressive  oversight  and  burden¬ 
some  inquiries  if  their  budgets  demonstrate  instabilities. 

However,  the  inherent  characteristics  of  a  SoS  will  likely  produce  less  stable 
budgets.  Division  of  programs  into  individual  budget  line  items  tends  to  emphasize 
unitary  goals,  as  opposed  to  the  pluralistic  goals  of  an  SoS.  Even  if  a  single  sponsor 
were  empowered  to  balance  limited  SoS  resources  across  an  array  of  component 
systems,  this  authority  would  probably  be  insufficient  to  compensate  “for  sudden  and 
potentially  dramatic  shifts  in  system  boundaries”  that  require  corresponding  resource 
shifts  (Keating  et  al.,  2003,  p.  41).  Furthermore,  Keating  et  al.  emphasize  that  budget- 
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ing  methodologies  to  operate  and  maintain  systems  of  systems  “once  they  have  been 
fielded  are  scarce”  (p.  43). 

Given  these  destabilizing  budget  pressures,  SoS  acquisition  will  face  daunt¬ 
ing  challenges  in  the  PPBE  system.  This  burden  may  aggravate  resourcing  tensions 
between  component  systems  and  jeopardize  system-wide  development  efforts.  A 
reasonable  and  effective  solution  for  this  problem  would  be  to  introduce  the  DAPA 
Panel’s  SPFA  concept  in  an  SoS,  empowering  its  lead  sponsor  to  manage  budget 
trade-offs  across  the  spectrum  of  component  systems. 

REQUIREMENTS:  DISCIPLINE  AND  FLEXIBILITY,  STRIKING  A  BALANCE  IN 
AN  SOS 

As  in  resourcing,  the  DAPA  panel  also  discovered  instabilities  in  requirements 
generation  that  were  rooted  in  differing  organizational  values.  Less  inclined  to  scruti¬ 
nize  details  of  program  management  and  funding,  requirements  writers  mandate  “sys¬ 
tems  that  are  technologically  unrealistic  or  unable  to  be  delivered  [on  time]”  ( DAPA 
Report,  2006,  p.  5).  Furthermore,  the  DAPA  Report  describes  how  extended  develop¬ 
ment  timeframes  destabilize  system  requirements  due  to  the  need  to  adapt  to  evolving 
operational  environments,  military  priorities,  acquisition  rules,  and  overarching  poli¬ 
cies.  Overall,  the  DAPA  panel  found  little  to  recommend  in  the  JCIDS,  characterizing 
the  system  as  cumbersome,  overly  complex,  and  unsuitable  for  continued  use. 


FIGURE  3.  DAPA  JOINT  CAPABILITIES  ACQUISITION  AND  DIVESTMENT 
PROCESS  AND  IMPLEMENTATION  PLAN 
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DAPA’s  Recommendation  on  Requirements.  The  DAPA  Panel  recommended 
scrapping  JCIDS  to  implement  a  new  system  (Figure  3).  Its  report  suggests  an  alter¬ 
native  Joint  Capabilities  Acquisition  and  Divestment  (JCAD)  planning  system  that 
would  streamline  and  simplify  processes  to  “enhance  requirements  stability”  ( DAPA 
Report,  2006,  p.  44).  Flowever,  features  of  the  JCAD  plan  that  emphasize  a  more 
linear  approach  and  increased  inter-Service  competition  might  interfere  with  effective 
SoS  requirements  generation. 

The  JCAD  system  would  expect  the  Joint  Staff  Director  for  Force  Structure, 
Resources  and  Assessment  (J-8)  to  develop  an  “integrated,  time-phased,  fiscally 
informed  capability,  acquisition,  and  divestiture  plan”  ( DAPA  Report,  2006,  p.  39). 
The  USD(AT&L)  would  then  use  the  J-8  plan  to  invite  competing  proposals  for  mate¬ 
rial  solutions  from  the  Services.  While  this  procedure  could  significantly  improve 
requirements  generation  for  many  programs,  it  could  introduce  counterproductive 
tensions  for  integrated  SoS  development. 

REQUIREMENTS  PROBLEMS  SPECIFIC  TO  SOS  DEVELOPMENT 

Developing  requirements  for  a  complete  SoS  will  likely  require  a  fresh  approach. 
Most  critics  of  the  JCIDS  rightly  criticize  its  inability  to  stabilize  requirements. 
However,  proposals  to  introduce  additional  discipline  in  requirements  generation 
fail  to  address  the  expected  challenges  of  SoS  development.  Instead,  SoSE  experts 
emphasize  the  need  for  increased  flexibility  to  cope  with  SoS  complexities. 


Developing  requirements  for  a  complete  SoS  will  likely 
require  a  fresh  approach . 


The  description  of  SoS  development  published  by  Keating  et  al.  (2003)  suggests 
elements  of  a  new  approach.  At  inception,  one  must  “proceed  with  the  assumption 
that  the  initial  problem  definition  or  mission  is  always  incorrect  or  suspect”  (p.  43). 
Additionally,  requirements  for  SoS  components  will  be  difficult  to  articulate  as 
“increasing  information  intensity,  contextual  richness,  and  problem  complexity  all 
contribute  to  the  need  for  evolving  systems  engineering  to  address  emergent  complex 
systems  problems”  (p.  40).  In  essence,  SoSE  requires  adaptation  to  changing  require¬ 
ments  beyond  what  current  acquisition  processes  allow. 

The  proposed  JCAD  system  does  not  provide  an  effective  construct  to  address 
these  complexities.  As  described  in  the  DAPA  Report  (2006),  JCAD  processes  are  lin¬ 
eal'  and  sequential.  This  process  is  ideally  suited  for  easily  described  capability  needs, 
but  will  not  facilitate  an  appreciation  for  complex  component  system  interactions  that 
contribute  to  SoS  system  capabilities. 
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SoSE  experts  envision  future  requirements  generation  as  a  fluid  process,  capable 
of  coping  with  shifting  environments.  Accordingly,  SoS  requirements  must  provide 
for  flexibility  in  parameters  and  boundaries.  Because  overall  SoS  performance  results 
from  complex  component  system  interactions,  the  J-8’s  ability  to  produce  an  “inte¬ 
grated,  time-phased,  fiscally  informed  capability,  acquisition,  and  divestiture  plan” 

(. DAPA  Report,  2006,  p.  39)  that  captures  SoS  capabilities  is  questionable.  Instead, 
the  J-8  should  describe  SoS  requirements  more  holistically.  The  proposed  JCAD  sys¬ 
tem  could  accommodate  such  an  approach  by  emphasizing  overarching  capabilities 
and  architectures,  rather  than  traditional  program  proposal  elements  such  as  system 
technical  descriptions,  delivery  profiles,  and  production  quantities. 

Furthermore,  the  JCAD  process  pits  Services  against  one  another  to  compete 
for  material  solutions.  This  competition  would  hinder  development  and  integration 
of  SoS  components  across  Service  boundaries.  As  the  JCAD  process  is  currently 
envisioned,  Services  competing  to  develop  SoS  components  would  lack  incentives  to 
prioritize  capabilities  that  enhanced  other  Services’  needs.  SoS  development  in  this 
parochial  manner  would  quickly  degenerate  into  inefficient  parallel  engineering  of 
systems  that  might  form  a  loosely  interoperable  system,  but  would  remain  a  far  cry 
from  the  capabilities  that  integrated  SoSE  could  achieve. 

Several  modifications  to  the  proposed  JCAD  could  tailor  its  processes  to  bet¬ 
ter  support  SoS  requirements  generation.  First,  because  SoS  development  must  be 
flexible  and  iterative,  the  J-8  should  describe  the  incremental  value  desired  for  new 
SoS  (or  particular  component)  capabilities  rather  than  attempt  to  create  an  integrated, 
time-phased  capability  plan  prior  to  program  inception.  Second,  material  solutions 
proposals  for  SoS  components  should  broadly  describe  capabilities  and  integrating 
architectures  instead  of  specific  technical  descriptions  and  force  structure  proposals. 
Finally,  inter-Service  competition  should  be  based  on  selecting  the  best  leader  for 
SoS  integration  efforts.  This  approach  would  facilitate  joint  cooperation  on  proposals, 
establishing  a  more  collaborative  atmosphere  for  SoS  development. 

CONCLUSION— RECOMMENDATIONS  TO  TAILOR  REFORM 

Decades  of  unsuccessful  reforms  have  frustrated  generations  of  professionals 
seeking  to  fix  defense  acquisition  through  incremental  improvements.  The  DAPA 
panel  proposes  to  break  this  pattern  with  a  sweeping  reform  plan,  actionable  recom¬ 
mendations,  and  an  implementation  strategy.  While  these  proposals  offer  significant 
promise,  they  do  not  anticipate  critical  challenges  presented  by  SoS  development. 

While  defense  leadership  is  counting  on  SoS  capabilities,  the  current  and  pro¬ 
posed  acquisition  systems  are  ill  equipped  to  facilitate  actual  development  of  these 
systems.  To  better  facilitate  SoS  acquisition,  the  following  modifications  to  the  DAPA 
Report  recommendations  are  suggested: 

■  Organization.  Plans  to  restructure  defense  acquisition  must  incorporate  mecha¬ 
nisms  to  coordinate  joint  efforts.  Assignment  of  a  lead-sponsor  to  oversee  inter- 
Service  collaboration  in  SoS  development  would  support  this  goal. 


188 


■  Budget.  Efforts  to  stabilize  acquisition  budgets,  such  as  the  SPFA,  should  be 
introduced  in  an  SoS  development  program.  This  arrangement  would  empow¬ 
er  an  SoS  lead-sponsor  to  make  trade-offs  among  component  systems. 

■  Requirements  Generation.  The  proposed  JCAD  processes  should  be  tailored 
to  three  specific  guidelines: 

a)  Focus  SoS  requirements  on  incremental  capability  needs  and  architectures 
instead  of  detailed  technical  descriptions. 

b)  Encourage  earlier  joint  cooperation  on  SoS  material  solutions. 

c)  Use  competition  to  select  SoS  lead-sponsorship  instead  of  Service-specific 
material  solutions  for  SoS  components. 

Incorporating  these  ideas  in  today’s  defense  acquisition  reforms  would  help 
catalyze  transformations  in  the  acquisition  workforce  needed  for  SoS  development  and 
significantly  improve  the  likelihood  that  tomorrow’s  systems  will  realize  their  potential. 
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DISCLAIMER 

The  views  expressed  in  the  article  are  the  author’s  and  do  not  reflect  the  official 
policy  or  position  of  the  Department  of  Defense  or  the  U.S.  Government. 


CDR  Scott  Moran,  USN,  is  a  naval  aviator  currently  serving  as 
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has  earned  aeronautical  engineering  degrees  from  MIT  and  Naval 
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ENDNOTES 


1.  SoSE  is  a  top-down,  comprehensive,  collaborative,  multidisciplinary,  iterative, 
and  concurrent  technical  management  process  for  identifying  system  of  systems 
capabilities;  allocating  such  capabilities  to  a  set  of  interdependent  systems; 

and  coordinating  and  integrating  all  the  necessary  development,  production, 
sustainment,  and  other  activities  throughout  the  life  cycle  of  a  system  of  systems. 
The  overall  objective  for  developing  a  system  of  systems  is  to  satisfy  capabilities 
that  can  only  be  met  with  a  mix  of  multiple,  autonomous,  and  interacting 
systems.  The  mix  of  constituent  systems  may  include  existing,  partially 
developed,  and  yet-to-be-designed  independent  systems  (DAG,  2006,  p.  100). 

2.  Many  variations  of  the  “little  a”  vs.  “big  A”  terminology  can  be  found,  but  nearly 
all  represent  the  same  basic  principle  discussed  here. 
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A  GLIMPSE  INTO  DoD 
WEAPON  SYSTEMS 
PROGRAMS 

Andy  Fainer 


An  important  strategic  topic  confronting  the  United  States  of  America  is 
sustaining  Department  of  Defense  (DoD)  weapon  systems  as  part  of  the 
overall  defense  life  cycle  management  process.  For  the  past  several  decades, 
billions  of  taxpayer  dollars  have  been  spent  on  weapon  systems  annually. 

The  lives  of  U.S.  Armed  Forces  members  and  the  people  they  protect  depend 
upon  the  quality  of  these  weapon  systems.  As  these  weapon  systems  have 
become  more  sophisticated  and  more  complex  coupled  with  a  decrease  in 
the  size  of  the  U.S.  Armed  Forces  over  the  past  30  years,  the  military  has 
become  increasingly  reliant  on  these  weapon  systems  for  our  nation's  security. 

This  article  focuses  on  three  major  aspects  of  the  topic:  (a)  background  informa¬ 
tion,  in  which  key  DoD  documents  and  concepts  such  as  the  2006  Quadrennial 
Defense  Review  (QDR),  2005  National  Defense  Strategy  and  logistics  trans¬ 
formation,  including  Future  Logistics  Enterprise,  will  be  discussed;  (b)  the  defense 
life  cycle  management  system  will  be  discussed;  and  (c)  an  analysis  of  the  defense 
life  cycle  management  system,  which  is  covered  by  six  Government  Accountability 
Office  (GAO)  reports. 


BACKGROUND 

DoD  is  the  executive  department  responsible  for  organizing  and  managing  all  of 
the  government’s  agencies  and  functions  relating  to  national  security  and  the  military. 
The  February  2006  QDR  serves  as  a  roadmap  for  change  to  transform  the  U.S.  mili¬ 
tary  to  an  outcome-oriented,  capability-based  force  to  better  support  the  joint  warf¬ 
ighter  well  into  the  21st  century.  It  calls  for  a  continuing  adaptation  and  reorientation 
for  an  integrated  joint  force  that  is  more  capable  to  defend  our  national  interests  as 
well  as  to  deter  and  defeat  our  adversaries.  The  QDR  shapes  DoD’s  plans,  policies, 
and  programs  into  a  broader  strategy  and,  later,  becomes  part  of  the  President’s 
budget  request. 
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The  cornerstone  of  the  2006  QDR  is  the  March  2005  National  Defense  Strategy. 
Although  the  U.S.  military  maintains  considerable  technological  advantages  in  the 
world,  our  adversaries  are  starting  to  catch  up.  Threats  are  categorized  as  traditional, 
irregular,  catastrophic,  and  disruptive.  A  traditional  challenge  uses  military  capabili¬ 
ties  and  forces  in  military  conflict.  An  irregular  challenge  uses  unconventional  meth¬ 
ods  to  offset  a  stronger  opponent’s  traditional  advantages.  A  catastrophic  challenge 
uses  weapons  of  mass  destruction  (WMD)  or  similar  methods  yielding  WMD-like 
effects.  A  disruptive  challenge  is  when  an  adversary  finds  a  breakthrough  technology 
that  nullifies  a  U.S.  advantage  (DoD,  March  2005). 


Although  the  U.S.  military  maintains  considerable 
technological  advantages  in  the  world,  our  adversaries  are 

starting  to  catch  up. 


Considering  these  four  threats,  the  QDR  identified  four  focus  areas:  (a)  defeat¬ 
ing  terrorist  networks,  (b)  defending  the  homeland  in  depth,  (c)  shaping  the  choices 
of  countries  at  strategic  crossroads,  and  (d)  preventing  hostile  states  and  non-state 
actors  from  acquiring  or  using  WMD  (DoD,  2006).  These  inter-related  focus  areas 
are  the  foundation  of  the  force  planning  construct,  showing  the  capabilities  and  forces 
needed  to  mitigate  the  four  threats  (DoD,  2006).  The  force  planning  construct  deter¬ 
mines  the  size  of  the  force  (capacity)  and  the  types  of  capabilities  (forces  and  equip¬ 
ment)  needed  for  a  diverse  range  of  scenarios  (DoD,  2006).  During  this  QDR,  senior 
leaders  confirmed  the  four  focus  areas,  but  divided  the  force  planning  construct  into 
three  objective  areas:  homeland  defense,  war  on  terror/irregular  warfare,  and  conven¬ 
tional  campaigns  (DoD,  2006). 

The  2006  QDR  listed  the  need  for  the  following  types  of  capabilities: 

■  Security  cooperation  and  engagement  activities  including  joint  training 
exercises,  senior  staff  talks,  and  officer  and  foreign  internal  defense  training  to 
increase  understanding,  strengthen  allies  and  partners,  and  accurately  com¬ 
municate  U.S.  objectives  and  intent.  This  will  require  both  new  authorities  and 
21st  century  mechanisms  for  the  interagency  process. 

■  Considerably  improved  language  and  cultural  awareness  to  develop  a  greater  un¬ 
derstanding  of  emerging  powers  and  how  they  may  approach  strategic  choices. 

■  Persistent  surveillance,  including  systems  that  can  penetrate  and  loiter  in 
denied  or  contested  areas. 

■  The  capability  to  deploy  rapidly,  assemble,  command,  project,  reconstitute,  and 
re-employ  joint  combat  power  from  all  domains  to  facilitate  assured  access. 
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■  Prompt  and  high- volume  global  strike  to  deter  aggression  or  coercion  and,  if 
deterrence  fails,  to  provide  a  broader  range  of  conventional  response  options 
to  the  President.  This  will  require  broader  authorities  from  the  Congress. 

■  Secure  broadband  communications  into  denied  or  contested  areas  to  support 
penetrating  surveillance  and  strike  systems. 

■  Integrated  defenses  against  short-,  intermediate-,  and  intercontinental-range 
ballistic  and  cruise  missile  systems. 

■  Air  dominance  capabilities  to  defeat  advanced  threats. 

■  Undersea  warfare  capabilities  to  exploit  stealth  and  enhance  deterrence. 

■  Capabilities  to  shape  and  defend  cyberspace. 

■  Joint  command  and  control  capabilities  that  are  survivable  in  the  face  of 
WMD-,  electronic-,  or  cyber-attacks  (DoD,  2006). 

The  2001  QDR  called  for  improved  effectiveness  and  efficiency  in  moving  and 
sustaining  military  forces  in  distant  theaters,  which  included  efforts  to  have  a  quicker 
deployment  process  and  reduce  the  logistics  footprint  and  its  associated  costs.  The 
2006  QDR  called  for  examining  supply  chain  logistics  costs  by  showing  the  relation¬ 
ship  between  resources  and  supply  chain  logistics  to  better  understand  the  costs  they 
incur.  Also,  it  called  for  continuous  performance  improvement.  For  example,  DoD 
uses  Radio  Frequency  Identification  (RFID)  to  carry  out  logistics  support  through 
automated  asset  visibility  and  management.  The  RFID  allows  for  the  sharing,  integra¬ 
tion,  and  synchronizing  of  data  from  the  strategic  to  the  tactical  level,  which  shows 
the  cause-and-effect  relationship  between  resources  and  readiness.  Using  RFID 
and  implementing  Lean  Six  Sigma  and  performance-based  logistics  will  markedly 
improve  DoD’s  supply  chain  (DoD,  2006). 

To  improve  sustainment  capability  by  achieving  an  integrated  joint  force  that 
is  more  agile  and  more  rapidly  deployable  with  a  reduced  logistics  footprint,  DoD 
adopted  the  Future  Logistics  Enterprise  (FLE).  It  serves  as  a  near-term  logistics  blue¬ 
print  to  strengthen  the  warfighter  from  2005  to  2010  through  end-to-end  customer 
service  and  enterprise  integration  into  the  21st  century  (FLE,  n.d.). 

The  FLE  has  six  interrelated  initiatives  to  meet  the  requirements  of  the  QDR  and 
the  National  Defense  Strategy. 

Depot  Maintenance  Partnership.  Depot  maintenance  services  cost  over  $17 
billion  annually.  The  puipose  of  this  initiative  is  to  increase  partnerships  with  the 
commercial  sector.  Due  to  national  security,  DoD  will  continue  to  retain  depot  main¬ 
tenance  capability,  but  will  encourage  increased  private  sector  investment  in  depot 
infrastructure,  better  facility  and  equipment  management,  and  better  depot  business 
practices  (FLE,  n.d.). 

Condition-Based  Maintenance  Plus  (CBM+).  Currently,  DoD  is  unable  to  pre¬ 
dict  equipment  failures  in  their  maintenance  programs,  resulting  in  excessive  supply 
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chain  costs.  The  purpose  of  this  initiative  is  to  improve  the  operational  availability 
and  readiness  of  weapon  systems  life  cycles  at  a  decreased  cost  by  using  improved 
maintenance  capabilities  and  integrated  logistics  and  business  processes  (FLE,  n.d.). 

Total  Life  Cycle  Systems  Management  (TLCSM).  Weapon  systems  sustainment 
uses  80  percent  of  DoD  logistics  resources  or  approximately  $64  billion  annually 
(Cothran,  Fowler,  &  Kratz,  2002).  DoD  is  changing  to  a  performance-based  weapon 
systems  sustainment  model  to  achieve  weapon  systems  performance  integration 
across  government  and  industry.  The  program  manager  (PM)  manages  and  is  ac¬ 
countable  for  the  development  of  the  weapon  system  and  is  responsible  for  meeting 
cost,  schedule,  and  performance  factors  while  considering  the  various  and  changing 
warfighter  performance  requirements  in  this  process.  Former  Assistant  Deputy  Under 
Secretary  of  Defense  for  Fogistics  Plans  and  Programs  Fouis  Kratz,  stated: 

One  of  the  fundamental  tenets  of  performance-based  logistics  is  the 
acquiring  of  weapon  system  support  as  an  integrated  package  based 
on  objective  outcomes,  such  as  system  availability.  The  objective 
outcomes — or  operational  performance  requirements  of  the  custom¬ 
er — will  be  documented  in  a  formal  performance  agreement  docu¬ 
ment,  negotiated  across  all  stakeholders,  consistent  with  the  Services’ 
corporate  structure.  The  performance  agreement  defines  system 
performance  expectations  (and  corresponding  support  required), 
resources  required  to  provide  that  level  of  performance,  commitment 
to  provide  those  resources,  and  signature  by  appropriate  stakeholders 
(P-51). 

End-to-End  Distribution.  The  purpose  of  this  initiative  is  to  enhance  the  flow 
of  materiel  to  the  end  user,  while  at  the  same  time,  synchronizing  deployment  and 
sustainment  efforts  into  an  integrated,  end-to-end  distribution  system  (Staff  Feature, 
2003). 

Executive  Agents  (EA).  The  purpose  of  this  initiative  is  to  ensure  that  EA  desig¬ 
nations  match  warfighter  requirements  with  the  National  Defense  Strategy  by  sup¬ 
porting  the  warfighter  “across  the  full  spectrum  of  operations/including  support  on  an 
end-to-end  basis  and  rapid  response  to  all  deployments”  (p.  3). 

Enterprise  Integration.  The  puipose  of  this  initiative  is  to  unite  information 
technologies  in  order  to  implement  new  logistics  business  processes. 

None  of  these  six  initiatives  can  function  by  itself.  Each  initiative  helps  the  other 
five  initiatives  by  building  an  integrated  logistics  enterprise. 

Kratz  stated  that  “the  most  powerful  weapon  in  the  world  is  useless  if  we  can’t 
deploy  and  use  it  effectively”  (Cothran,  Fowler,  &  Kratz,  2002,  p.  50).  Fogistics 
transformation  serves  the  warfighter  threefold:  (a)  to  adopt  the  best  business  prac¬ 
tices,  (b)  to  have  a  logistics  system  open  architecture  so  decision  makers  can  use 
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integrated  logistics  information,  and  (c)  to  better  logistics  responsiveness  to  the  joint 
warfighter  (Staff  Feature,  2003).  As  outlined  in  the  2006  QDR,  the  United  States 
faces  several  types  of  dangerous  threats.  Logistics  transformation,  including  the 
Future  Logistics  Enterprise,  is  needed  in  order  to  maintain  technologically  superior 
weapon  systems  for  our  well-deserving  warfighter.  Logistics  matters  have  often  been 
crucial  in  determining  the  outcome  of  wars. 

DEFENSE  LIFE  CYCLE  MANAGEMENT  SYSTEM 

The  2006  QDR  described  the  need  for  continued  transformation  of  acquisition 
and  logistics  processes  in  order  to  be  more  agile  and  more  expeditionary,  to  increase 
reliability  of  DoD  weapon  systems,  and  to  reduce  logistics  footprint.  The  defense 
life  cycle  management  system  is  a  total  life  cycle  management  system.  Established 
by  Department  of  Defense  Instruction  5000.2,  Operation  of  the  Defense  Acquisition 
System,  in  May  2003  as  a  revised  policy,  the  defense  life  cycle  management  system 
is  a  knowledge-based,  phased,  evolutionary  process  used  for  the  acquisition  of  major 
defense  weapon  systems. 


The  defense  (requisition  system  uses  a  streamlined 
management  profess  that  delivers  tapable,  reliable ,  and 
sustainable  systems  to  the  user. 


DoD  has  three  major  decision-making  support  systems  to  support  the  overall 
defense  life  cycle  management  system:  (a)  Planning,  Programming,  Budgeting  &  Ex¬ 
ecution  (PPBE);  (b)  Joint  Capabilities  Integration  and  Development  System  (JCIDS); 
and  (c)  Defense  Acquisition  System.  These  three  systems  use  an  integrated  approach 
for  strategic  planning,  identification  of  military  capabilities’  needs,  system  acquisi¬ 
tion,  and  program  and  budget  development.  The  three  support  systems  work  together 
simultaneously  while  the  phases  are  carried  out  throughout  the  life  cycle. 

The  PPBE  is  DoD’s  primary  resource  allocation  process.  The  PPBE  has  three  ob¬ 
jectives.  First,  it  is  a  biennial,  calendar-driven  process  used  for  obtaining  funding  for 
major  weapon  systems  acquisition.  Second,  it  provides  a  factual  basis  for  affordabil¬ 
ity  assessment  and  resource  allocation  decisions.  Third,  it  offers  a  formal  structured 
system  for  making  decisions  on  policies,  strategies,  prioritized  goals,  and  the  devel¬ 
opment  of  necessary  forces  and  capabilities  to  accomplish  DoD’s  various  missions. 

The  JCIDS  is  dependent  on  warfighting  deficiencies  or  needs.  It  assesses  mission 
requirements  and  strategies  for  meeting  those  requirements  as  well  as  providing  the 
basis  for  establishing  priorities  (DAU,  ACQ-101,  Lesson  2). 

The  defense  acquisition  system  uses  a  streamlined  management  process  that 
delivers  capable,  reliable,  and  sustainable  systems  to  the  user.  It  is  an  event-driven, 
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risk  management  process  that  uses  periodic  reviews  and  program  approvals  to  prog¬ 
ress  into  subsequent  efforts  of  the  acquisition  life  cycle.  The  process  also  connects 
milestone  decisions  to  demonstrated  accomplishments. 


FIGURE  1.  THE  DEFENSE  ACQUISITION  MANAGEMENT  FRAMEWORK 


Process  entry  at  Milestone  A,  B,  or  C 

Entrance  criteria  met  before  entering  phase 
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The  framework,  as  depicted  in  the  framework  chart  shown  here,  is  divided  into 
three  activities:  Pre-Systems  Acquisition,  Systems  Acquisition,  and  Sustainment. 
These  activities  are  divided  into  five  phases:  Concept  Refinement,  Technology 
Development,  System  Development  and  Demonstration  (SDD),  Production  and  De¬ 
ployment,  and  Operations  and  Support.  A  milestone  or  decision  point  comes  before 
each  phase.  The  concept  decision  approves  entry  into  the  Concept  Refinement  Phase. 
Milestone  A  approves  entry  into  the  Technology  Development  phase.  Milestone  B 
approves  entry  into  the  SDD.  The  Design  Readiness  Review  approves  entry  into  the 
System  Demonstration  phase.  Milestone  C  approves  entry  into  the  Production  and 
Deployment  Phase.  The  Full-Rate  Production  Review  authorizes  Full  Rate  Produc¬ 
tion.  The  two  activities  of  Systems  Acquisition  and  Sustainment  are  divided  into  six 
work  areas:  System  Integration,  System  Demonstration,  Low  Rate  Initial  Production, 
Full-Rate  Production  and  Deployment,  and  Sustainment  and  Disposal. 

The  acquisition  process  begins  with  a  selected  concept  to  meet  a  particular 
capability  need.  In  the  Concept  Decision  review,  the  Milestone  Decision  Authority 
(MDA)  determines  entry  into  the  Concept  Refinement  phase.  Also,  a  date  for  Mile¬ 
stone  A  is  established  and  an  Acquisition  Decision  Memorandum  (ADM)  is  prepared 
and  documents  the  decision’s  results.  Entry  into  the  first  phase  does  not  signify  a  new 
acquisition  will  begin. 

ANALYSIS  OF  THE  DEFENSE  LIFE  CYCLE  MANAGEMENT  SYSTEM 

The  2006  QDR  stated  there  is  an  increasing,  profound  concern  among  DoD’s 
senior  leadership  and  Congress  or,  more  specifically,  the  GAO,  about  the  Major  De¬ 
fense  Acquisition  Programs  (MDAP).  This  lack  of  confidence  is  a  result  of  measuring 
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weapon  systems  acquisition  by  cost,  schedule,  and  performance.  These  acquisition 
programs  are  unpredictable  and  unstable.  Ongoing  reviews  for  acquisition  improve¬ 
ments  are  being  conducted  both  within  and  outside  DoD  to  enforce  these  acquisi¬ 
tions,  resulting  in  better  outcomes  for  the  taxpayer  and  more  responsive  support 
to  the  joint  warfighter  in  the  21st  century.  The  major  problems  addressed  by  GAO 
reports  are  summarized  below. 

First,  the  United  States  has  the  world’s  best  weapon  systems,  unrivaled  in  supe¬ 
riority.  The  process  to  deploy  these  systems  needs  to  be  fixed.  In  today’s  acquisition 
environment,  40  percent  cost  increases  are  common,  which  add  up  to  tens  of  mil¬ 
lions  of  dollars,  schedule  delays  that  add  up  to  years,  and  rebaselining  of  some  large 
and  expensive  programs;  indeed,  some  programs  are  even  scrapped.  Consequently, 
reduced  quantities  and  capabilities  are  delivered  to  the  warfighter. 

For  the  past  35  years  GAO  has  documented  these  problems.  Since  DoD  will  be 
spending  more  than  $1.4  trillion  dollars  for  new  weapon  systems  between  2005  and 
2009,  quality  and  time  are  essential  to  maintain  weapons  superiority,  quickly  counter 
threats  from  the  nation’s  adversaries,  and  better  protect  and  enable  the  warfighter. 
Also,  using  constant  2006  dollars,  the  top  five  programs  in  2001  cost  $290.8  billion 
while  the  top  five  programs  in  2006  totaled  $550  billion.  Considering  this  staggering, 
increased  dollar  spending  for  weapon  systems,  the  nation  will  be  faced  with  taking 
funds  from  other  federal  programs  to  fund  these  systems  or  to  reduce  funding  for 
these  systems. 

Second,  DoD  does  not  separate  long-term  needs  from  wants.  DoD  starts  many  pro¬ 
grams  that  it  cannot  afford.  Each  Service  competes  for  funding  creating  a  Service-cen¬ 
tric  structure  and  fragmented  decision-making  approach.  It  does  not  prioritize  programs 
based  on  customer  needs  and  DoD’s  long-term  vision.  Many  times  when  a  program 
needs  funds  because  of  cost  increases  or  schedule  delays,  funds  are  taken  away  from 
other  programs.  This  rewards  poor  performing  programs  (GAO,  March  2007). 

An  integrated  portfolio  management  investment  strategy,  as  used  successfully  in 
the  commercial  world,  would  achieve  more  executable  programs  and  ensure  better 
return  on  investments.  More  importantly,  warfighters  would  receive  greater  quanti¬ 
ties  and  capabilities  as  promised  to  them.  An  investment  strategy  would  prioritize  the 
order  of  needed  capabilities  and  match  them  up  against  resources — dollars,  technolo¬ 
gies,  time,  and  people  required  to  obtain  these  capabilities  as  well  as  define  incre¬ 
mental  product  development  programs  for  obtaining  these  capabilities  and  establish 
controls  so  the  requirements,  funding,  and  acquisition  processes  would  work  together. 
Without  an  integrated  investment  strategy,  all  other  improvements  will  fail  as  shown 
in  the  past  (GAO,  March  2007). 

Third,  in  2003  DoD  adopted  the  defense  life  cycle  management  system.  This  is 
a  knowledge-based,  evolutionary  product  development  approach.  GAO  examined 
programs  that  began  after  this  system  started.  DoD  is  not  following  it.  Early  decision 
points  in  the  system  are  frequently  bypassed,  resulting  in  decision  makers  committing 
programs  to  premature  system  demonstration  and  initial  manufacturing  in  the  face  of 
significant  unknowns  about  technology,  design,  and  production. 

Critical  points  such  as  Milestones  A  and  B,  the  Design  Readiness  Review,  and 
Milestone  C  are  not  followed  at  all  or  only  partially.  Programs  start  system  develop- 
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TABLE  1.  TECHNOLOGY  MATURITY  AND  PROGRAM  OUTCOMES 


Program 

Percent  increase  in 

R&D  (first  full  estimate 
to  latest  estimate) 

Percent  of  critical 
technologies  and 
associated  maturity 
level  at  development 
start 

Advanced  Threat  Infrared 
Countermeasure/Common 
Missile  Warning  System 

5.6 

50%  (3  of  6)  at  6 
or  higher 

C-5  Reliability  Enhancement 
and  Reengining  Program 

2.1 

100%  (11  of  11)  at  6 
or  higher 

DD(X)  Destroyer 

417.3 

25%  (3  of  12)  at  6 
or  higher 

Future  Combat  System 

50.8 

32%  (17  of  52)  at  6 
or  higher 

Joint  Strike  Fighter 

30.1 

25%  (2  of  8)  are  6 
or  higher 

(GAO,  2005,  p.  42) 


TABLE  2.  PERCENT  OF  PROGRAMS  THAT  ACHIEVED  TECHNOLOGY 
MATURITY  AT  KEY  JUNCTURES 

80 
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0 


(GAO,  2006,  p.  11) 


Development  start  DoD  design  review  Production  decision 


ment  with  immature  technologies  (DoD,  April  2006).  The  Army’s  Future  Combat 
System  entered  the  SDD  phase  with  32  percent  of  its  critical  technologies  mature. 

The  Air  Force’s  Joint  Strike  Fighter  began  SDD  with  only  25  percent  mature  technol¬ 
ogies.  When  programs  start  development  with  mature  technologies,  they  incur  lower 
development  and  unit  cost  increases  than  those  programs  starting  with  immature 
technologies.  Table  1  shows  five  examples. 

Fhstory  shows  that  programs  with  lower  levels  of  knowledge  at  critical  points  lack 
demonstrated  knowledge  in  the  process  and  will  continue  to  stay  behind  (Table  2). 
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TABLE  3.  COST  AND  SCHEDULE  OUTCOMES  FOR  6  OF  THE  10 
LARGEST  DEVELOPMENT  PROGRAMS  SORTED  BY  PERCENT  OF  SYSTEM 
DEVELOPMENT  REMAINING 


Programs 

Percent 
development 
cost  growth 

Delay  in 
delivery 
of  initial 
capability  in 
months 

Percent  of 

development 

remaining 

Aerial  Common 

Sensor 

45% 

24 

85% 

Future  Combat 

System 

48% 

48 

78% 

Joint  Strike  Fighter 

30% 

23 

60% 

Expeditionary 

Fighting  Vehicle 

61% 

48 

49% 

C-130  Avionics 
Modernization 

Program 

122% 

Delays 
anticipated 
due  to  program 
restructure 

Undetermined 
due  to  program 
restructure 

Delays 
anticipated 
due  to  program 
restructure 

Global  Hawk  (RQ-4B) 

166% 

Delays 
anticipated 
due  to  program 
restructure 

Delays 
anticipated 
due  to  program 
restructure 

(GAO,  2006,  p.  10) 


Fourth,  Tables  3  and  4  show  that  immature  technologies  that  advance  without 
adequate  demonstrated  knowledge  result  in  cost  increases  and  schedule  delays;  thus, 
DoD  delivers  reduced  quantities  and  capabilities  to  the  warfighter  and,  ultimately, 
loses  buying  power. 

Table  4  gives  examples  of  DoD  programs  with  reduced  buying  power  (GAO, 
April  2006). 

Fifth,  to  improve  the  defense  life  cycle  management  system,  an  item’s  technical 
data  must  be  included  and  readily  available.  Technical  data  are  recorded  information 
used  to  define  a  design  and  to  produce,  support,  maintain,  or  operate  the  item  (GAO, 
July  2006).  These  data  represent  a  significant  element  in  the  life  cycle  of  a  weapon 
system  because  they  may  be  used  for  decades.  The  Army  and  the  Air  Force  have  ex¬ 
perienced  sustainment  limitations  on  several  deployed  weapon  systems  because  they 
lacked  needed  technical  data  rights.  They  have  not  been  able  to  take  advantage  of  cost 
savings  and  meet  legislative  requirements  for  depot  maintenance.  GAO  pointed  out 
seven  weapon  systems  lacking  technical  data  rights:  C-17,  F-22,  and  C130J  aircraft, 
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TABLE  4.  EXAMPLES  OF  DOD  PROGRAMS  WITH  REDUCED  BUYING 
POWER 


Programs 

Initial 

Estimate 

Initial 

quantity 

Latest 

estimate 

Latest 

quantity 

Percent 
of  unit 
cost 
increase 

Joint  Strike 
Fighter 

$189.6  B 

2,866 

aircraft 

$206.3  B 

2,458 

aircraft 

26.7% 

Future  Combat 
Systems 

$82.6  B 

1 5  systems 

$127.5  B 

15 

systems 

54.4% 

F-22A  Raptor 

$81.1  B 

648  aircraft 

$65.4  B 

181  aircraft 

188.7% 

Evolved 
Expendable 
Launch  Vehicle 

$15.4  B 

181 

vehicles 

$28.0  B 

138 

vehicles 

137.8% 

Space  Based 
Infrared  System 
High 

$4.1  B 

5  satellites 

$10.2  B 

3  satellites 

315.4% 

Expeditionary 
Fighting  Vehicle 

$8.1  B 

1,025 

vehicles 

$11.0  B 

1,025 

vehicles 

35.9% 

(GAO,  2006,  p.  5) 


the  Up-armored  High-Mobility  Multipurpose  Wheeled  Vehicle,  the  Striker  family 
of  vehicles,  the  Airborne  Warning  and  Control  System  aircraft,  and  the  M4  carbine 
(GAO,  July  2006).  Long-term  technical  data  need  to  be  in  the  requirements  before 
contract  solicitation  issuance  (GAO,  July  2006). 

Sixth,  the  definition  of  success  needs  redefining.  In  the  commercial  world,  suc¬ 
cess  is  defined  by  maximizing  profit.  At  DoD,  success  is  defined  by  the  ability  to  ob¬ 
tain  funds  for  new  programs  and  to  maintain  funding  for  current,  ongoing  programs. 
Optimistic  cost,  schedule,  and  technology  readiness  factors  attract  funding.  Honest 
assessments  could  result  in  a  loss  of  funding.  Delayed  testing  is  preferred  over  early 
testing  because  bad  news  could  result  in  a  loss  of  funds.  Success  measures  such  as 
risk  reduction,  knowledge-based  decision  making,  discipline,  collaboration,  trust, 
commitment,  consistency,  realism,  and  accountability  could  result  in  better  outcomes 
for  DoD  (GAO,  2005). 


CONCLUSION 

DoD’s  acquisition  of  major  weapon  systems  represents  one  of  the  most  cru¬ 
cial  and  expensive  activities  in  the  federal  government.  Its  impact  is  critical  on  the 
nation’s  economic  and  fiscal  policies,  especially  considering  the  current  long-term 
fiscal  imbalances  along  with  increasing  conflicts  over  increasingly  scarce  resources. 
This  could  damage  our  national  security.  For  the  past  several  decades,  deep  concern 
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over  DoD’s  management  effectiveness  of  these  weapon  systems  is  prompting  DoD 
to  change  business  practices.  By  following  DoD’s  defense  life  cycle  management 
system,  each  weapon  system  could  be  an  affordable,  worthwhile  investment  and  an 
executable  program  and,  thus,  achieve  a  better  acquisition  outcome.  DoD  is  trans¬ 
forming  military  operations  to  function  as  a  joint  force  on  the  battlefield  in  accor¬ 
dance  with  the  2006  QDR. 


DISCLAIMER 

The  views  expressed  in  this  article  are  the  author's  and  do  not  reflect  the  official 
policy  or  position  of  Information  Technology,  E-Commerce,  and  Commercial 
Contracting  Center  (ITEC4-W). 
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